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ABSTRACT 

Though  one  is  led  to  believe  that  program  transformation  systems  which  perform  source-to- 
source  transformations  enable  the  user  to  understand  and  appreciate  the  resulting  source  program, 
this  is  not  always  the  case.  Transformations  are  capable  of  behaving  and/or  interacting  in  unex¬ 
pected  ways.  The  user  who  is  interested  in  understanding  the  whats,  whys,  whcrcs,  and  hows  of  the 
transformation  process  is  left  without  tools  for  discovering  them. 

I  provide  an  initial  step  towards  the  solution  of  this  problem  in  the  form  of  an  accountable 
sourcc-to-sourcc  transformation  system.  It  carefully  records  the  information  necessary  to  answer 
such  questions,  and  provides  mechanisms  for  the  retrieval  of  this  infonnation.  It  is  observed  that 
though  this  accountable  system  allows  the  user  access  to  relevant  facts  from  which  he  may  draw 
conclusions,  further  study  is  necessary  to  make  the  system  capable  of  analyzing  these  facts  itself. 
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ac-count-a-ble  adj.  I  :  subject  to  giving  an  account 
2  :  capable  of  being  accounted  for 

ac-count  vb.  vl  /  :  to  give  a  report  on 

vi  I  :  to  furnish  a  justifying  analysis  or  explanation  -  used  with  for 

—Webster's  Seventh  New  Collegiate  Dictionary 


We  might  suppose  that  by  the  combined  action  of  appropriate  forces 
any  material  form  could  be  transformed  into  any  other:  just  as  out  of 
a  'shapeless'  mass  of  clay  the  potter  or  the  sculptor  models  his  artistic 
product:  or  just  as  we  attribute  to  Nature  herself  the  power  to  effect 
the  gradual  and  successive  transformation  of  the  simple  germ  into  the 
complex  organism.  ...  So  the  living  and  the  dead,  things  animate  and 
inanimate,  we  dwellers  in  the  world  and  this  world  in  which  we  dwell 
...  are  bound  alike  by  physical  and  mathematical  law. 

—D’Arcy  Tho.npson 


Chapter  One 

Source-To-Source  Transformation  Systems 


Program  TRANSFORMATION  SYSTEMS  have  been  produced  which  claim  to  ameliorate  the 
problems  of  inefficient  programs  caused  by  modular  programming  strategies,  initial  lack  of 
attention  to  efficiency  (for  indeed,  during  the  programming  process,  construction  and  verification 
should  come  first),  or  perhaps  just  bad  programming  style.  T  hese  transformation  systems  may 
be  implemented  automatically  or  manually,  and  may  do  their  work  at  compile  time  or  before. 
For  example,  an  optimizing  compiler  is  an  automatic  transformation  system  which  performs  its 
work  at  compile  time.  In  contrast,  a  source-to-source  transformation  system  is  an  automatic  pre¬ 
processor  whose  output  may  then  be  compiled  on  the  desired  machine.  Alternatively,  the  program¬ 
mer  himself  may  be  responsible  for  producing  cfficie  t  code,  and,  following  the  construction  and 
verification  of  his  program,  must  rewrite  any  part  which  docs  not  meet  its  operational  standards.  In 
this  chapter  I  offer  my  opinions  and  suggestions  regarding  the  current  suite  of  program  transforma¬ 
tion. 


1.1,  Introduction 

Program  transformation  should  be  a  process  which  is  independent  of  compilation,  and  which, 
though  performed  automatically,  can  be  guided  by  the  programmer.  A  source-to-source  transfor¬ 
mation  system  provides  a  programmer  with  canned  sets  of  transformations  which  he  may  invoke 
and  afterwards  observe  the  effect  of.  I  am  interested  in  a  more  interactive  approach  in  which  the 
programmer  is  able  to  intelligently  select  those  transformations  which  are  relevant  and  effective 
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for  the  particular  situation  he  presents.  Furthermore,  the  programmer  may  designate  only  certain 
parts  of  the  program  for  transformation  (for  example,  he  may  choose  to  apply  optimizing  transfor¬ 
mations  to  code  which  forms  a  bottleneck  at  execution  time).  In  order  to  intelligently  make  these 
decisions,  the  programmer  must  have  the  freedom  to  experiment  with  various  sets  rtf  transforma¬ 
tions  to  his  program,  be  able  to  observe  their  outcomes,  and  verify  that  indeed  they  do  the  job  he 
requires.  The  advantages  of  tins  approach  arc  discussed  in  [I.oveman  1977). 

Although  source-to-sourcc  transformation  systems  allow  programmers  to  perform  an  analysis 
of  the  results,  they  provide  no  mechanisms  for  such  analyses;  the  programmer  must  study  the 
output  himself  to  discover  what  transformations,  if  any.  occurred.  Furthermore,  the  elfort  needed 
to  understand  why  a  transformation  applied  (or  more  often,  why  one  did  not)  is  significant  and 
prohibitive.  Without  mechanisms  to  aid  the  programmer  in  his  study  of  the  result  of  the  transfor¬ 
mation  process,  many  of  the  advantages  of  a  source-to-sourcc  transformation  system  arc  lost. 

Since  I  had  to  develop  a  sourcc-to-source  transformation  system  one  summer  without  die  aid 
of  any  sort  of  mechanism  for  evaluating  the  effectiveness  of  transformations,  I  have  a  great  many 
suggestions  to  offer  as  to  what  sorts  of  tools  would  be  valuable  for  doing  so.  It  was  often  necessary 
first  to  learn  if  there  had  been  any  change  in  the  argument  program  at  all.  that  is,  if  any  transforma¬ 
tion  had  succeeded  in  apply  ing.  A  transformation  system  which  would  report  whether  or  not  it  has 
done  anything  w  ould  be  capable  of  providing  this  information.  Then,  once  it  had  been  determined 
that  something  had  happened,  it  was  necessary  to  discover  what.  Ihcn  where,  then  when,  then  why. 
And  there  was  always  why  not?  I  w  ished  I  had  a  transformation  system  which  would  automatically 
account  for  its  decisions:  an  accountable  source-to-sourcc  transformation  system. 


I.I.I.  Statement  of  Thesis 

•  Sourcc-to-sourcc  transformation  systems  which  produce  changes  in  the  user's  program  without 
any  explanation  or  justification  of  these  changes  in  some  sense  "violate”  the  user's  code. 
To  verify  that  the  transformations  performed  were  those  expected  or  desired,  die  user  must 
manually  examine  the  resultant  code.  Furthermore,  understanding  why  such  transformations 
were  or  were  not  performed  requires  manual  examination  of  the  transformation  system  itself. 

•  A  source-to-sourcc  transformation  system  which  can  be  queried  as  to  what,  where,  when,  and 
why  it  did  whatever  it  did  can  aid  die  interactive  transformation  of  a  program. 

•  The  development  of  such  a  tool  is  feasible. 


].1.2.  Organization  of  Dissertation 

So  far  in  diis  chapter  I  have  given  an  overview  of  die  problem  I  wish  to  address,  and  only 
a  hint  of  the  proposed  solution.  In  the  remainder  of  this  chapter  I  will  discuss  the  uses  for  and 
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advantages  of  sourcc-to-sourcc  transformation  systems,  then  go  over  again  in  some  detail  (and 
with  examples)  why  many  of  those  advantages  are  not  realized.  Chapter  Two  describes  a  method 
for  regaining  these  advantages:  making  the  system  accountable  for  its  actions.  In  that  chapter  1 
will  define  exactly  what  1  mean  by  an  accountable  transformation  system,  then  present  the  design 
of  such  a  system.  Chapter  Three  begins  our  discussion  of  the  implementation  of  the  system  by 
introducing  the  sourcc-to-sourcc  transformation  system  which  will  be  made  accountable.  Finally, 
Chapter  Four  follows  the  decisions  and  revisions  involved  in  implementing  an  accountable  source- 
to-sourcc  transformation  system.  The  bulk  of  the  thesis  lies  in  this  chapter,  since  only  in  attempt¬ 
ing  to  implement  an  accountable  system  did  many  of  the  subtle  difficulties  in  doing  so  present 
themselves. 

In  this  dissertation  I  am  particularly  interested  in  discussing  source- to- source  transformation 
systems,  although  I  admit  that  if  someone  wished  to  argue  the  point,  I  would  have  a  hard  time 
convincing  him  there  was  any  difference  in  transformation  systems  which  are  sourcc-to-sourcc  and 
those  that  arc  not.  What  is  object  code  to  one  compiler  may  be  source  code  to  another;  however,  I 
will  assume  the  popular  definition  of  source  code  as  a  high-level  language,  and  make  no  claims  as 
to  whether  or  not  it  is  object  code  as  well,  in  this  discussion,  then,  it  will  be  assumed  that  when  1 
speak  of  transformation  systems,  I  mean  source- to- source  transformation  systems,  unless  explicitly 
stated  otherwise. 


1.2.  Motivation  And  Uses  For  Transformation  Systems 

Transformations  allow  one  to  automate  almost  any  kind  of  systematic  change  to  a  program. 
While  some  systems  arc  written  with  a  particular  transformation  set  “built-in”  because  they  are 
meant  to  pci  form  a  specific  task,  others  arc  written  more  modularly  and  may  be  used  with  any 
of  a  number  of  different  canned  transformation  sets.  General-purpose  transformation  systems 
exist  as  well;  these  come  with  a  special  language  in  which  the  programmer  himself  can  write  the 
transformations  he  wishes  to  be  applied  to  his  program. 

Sourcc-to-sourcc  transformation  systems  provide  access  to  the  resultant  code  in  a  way  that 
other  transformation  systems  do  not.  The  output  from  such  systems  may  be  studied,  modified, 
or  run  through  die  transformation  system  again  before  any  compilation  is  done.  Thus,  whether 
the  programmer  uses  a  set  of  canned  transformations  or  “rolls  his  own,”  lie  is  able  to  afterwards 
observe  the  effects  of  each  on  the  input  code. 

Sourcc-to-sourcc  transformations  may  be  applied  to  a  program  independently  of  the  machine 
the  program  is  going  to  be  compiled  on.  Since  the  system  takes  source  code  and  produces  source 
code,  transforming  die  program  precedes  any  compiler  activity.  Thus  the  programmer  has  the 
security  of  knowing  that  he  can  produce  a  more  efficient  program  for  any  machine.  Of  course, 
certain  transformations  may  be  more  or  less  effective  depending  on  the  target  machine.  Hie  pro- 
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grammcr  who  is  aware  of  these  differences  may  tailor  his  program  to  the  appropriate  machine,  or  to 
several  by  generating  all  of  the  various  versions  that  are  required. 

Currently  available  sourcc-to-sourcc  transformation  systems  are  capable  of  performing  op¬ 
timization,  realization,  and  translation. 


1.2.1.  Optimization 

By  far  the  most  common  use  of  transformation  systems  is  for  the  optimization  of  code. 
A  number  of  papers  which  describe  standard  optimizing  transformations  have  been  published 
[Standish,  ct  al  1976]  [Allen  and  Cocke  1972],  and  some  successful  systems  are  in  use  [Kerns  1977] 
[Boyle  and  Matz.  1977]  [Atkinson  1976],  No  one  will  question  die  need  for  program  optimization, 
but  what  arc  die  advantages  of  perf  nning  optimizations  by  means  of  a  transformation  system? 


1.2. 1.1.  Opens  the  Way  For  Interactive  Optimization 

Code  generated  by  a  sourcc-to-sourcc  transformation  system  may  be  analyzed  by  die  pro¬ 
grammer  in  order  to  determine  what  optimizations  were  performed.  He  is  diercforc  able  to  un¬ 
derstand  to  some  extent  exactly  what  transformations  to  the  code  arc  causing  any  difference  in 
execution  time.  He  can  see  which  parts  of  the  code  arc  being  modified,  and  which  arc  not.  On 
the  basis  of  these  observations,  he  may  write  or  use  other  transformations,  or  choose  other  pieces 
of  code  to  optimize.  Because  he  has  a  better  understanding  of  die  optimization  process,  he  can 
diercforc  better  control  that  process. 


1.2. 1 .2.  Simplifies  the  Compilation  Task 

By  moving  die  optimization  process  out  of  die  compilation  process,  a  compiler  writer  is  free 
to  concentrate  on  code  generation.  In  modularizing  the  entire  program  development  process  and 
allowing  compilation  to  be  independent  of  optimization,  we  move  closer  to  the  possibility  of 
automatic  generation  of  compilers. 

As  transformation  systems  evolve,  one  can  imagine  canned  sets  of  transformations  becoming 
available  which  arc  guaranteed  to  cure  all  programming  ills,  bach  set  of  transformations  may 
be  keyed  for  use  with  a  particular  language,  depending  on  the  target  machine.  For  example,  all 
Fortran  programs  to  be  run  on  an  IBM  370/165  may  use  transformation  set  “IKII370FIXIT",  while 
l  isp  code  to  be  run  on  a  DKC-10  system  will  use  ^information  set  "BA/.Ol.A".  Fach  set  would 
know  about  the  quirks  specific  to  its  target  machine,  and  be  prepared  to  put  the  source  code  into 
die  form  best  suited  for  efficient  compilation  on  that  machine. 
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Or,  better  yet.  let  there  be  only  one  transformation  set  for  each  language.  The  Fortran  set  is 
responsible  for  optimizing  Fortran  code,  the  l.isp  set  for  l.isp  code,  et  cetera.  Hach  set  of  transfor¬ 
mations  puts  its  source  code  into  a  form  suitable  for  efficient  compilation.  That  is,  there  should  be 
a  canonical  set  of  transformations  that  every  compiler  expects  to  have  already  been  performed  by 
the  time  the  compiler  gets  the  source  code.  Since  every  high-level  language  is  not  equally  capable 
of  expressing  all  the  possible  optimizations  of  a  piece  of  code,  I  will  instead  say:  for  each  high-level 
language,  there  should  be  a  canonical  set  of  transformations  that  every  compiler  for  that  language 
expects  to  have  already  been  performed.  Then,  it  is  the  responsibility  of  the  compiler  to  know  the 
quirks  and  idiosyncrasies  of  its  machine,  and  to  generate  efficient  object  code  for  that  machine.  The 
compiler  may  assume  that  the  source  code  it  receives  has  been  optimized  into  the  canonical  form 
agreed  upon  beforehand. 


1.2.2.  Realization 

Although  transformations  which  perform  optimizations  to  a  program  have  received  more  at¬ 
tention  than  those  which  do  not,  other  uses  for  transformation  systems  should  not  be  overlooked. 
One  transformation  system  currently  in  use  at  Argonne  National  laboratory  (Hoyle  and  Matz  1977J 
has  a  library  of  transformation  sets  which  generate  different  versions  or  ‘‘realizations’’  of  a  single 
prototype  program.  For  example,  given  a  I'ortran  program  to  perfonn  some  algorithm  upon  com¬ 
plex  numbers  in  single  precision,  it  can  generate  an  analogous  program  which  performs  the  same 
algorithm  on  real  numbers  in  double  precision.  This  is  done  by  first  applying  a  set  of  complex-to- 
real  transformations  to  the  program,  and  then,  using  the  output  of  that  run  as  input  to  the  next, 
applying  a  set  of  single-to-double  precision  transformations. 

This  method  may  be  used  to  generate  realizations  of  a  program  which  arc  geared  to  run  more 
efficiently  (or  perhaps  correctly!)  on  a  particular  machine.  The  notion  of  providing  portability  and 
reliability  by  program  transformation  is  discussed  in  [Hoyle  1976].  A  particular  realization  can  be 
thought  of  as  a  refinement  of  the  prototype  program:  transformations  may  be  written  to  refine 
high-level  programs  into  underlying  representations  (Standish,  et  al  1976]. 


1.2.3.  Translation 

Transformations  sets  may  also  be  written  to  translate  code  from  one  source  language  to 
another  [Pitman  1979],  These  transformations  arc  clearly  non-trivial,  and  the  set  of  programs  to 
which  they  can  successfully  apply  may  be  very  restricted  if  the  languages  arc  not  similar.  Syntax 
transformations,  however,  should  be  simple  and  fairly  straightforward. 
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1.3.  Inadequacies  of  Transformation  Systems 


Although  in  theory  a  source-to-sourcc  transformation  system  prov  ides  all  the  advantages  given 
above,  in  practice  things  turn  out  a  little  differently.  Consider  the  following  example: 

A  set  of  optimizing  transformations  was  applied  to  a  simple  MSP  program  to  compute  two 
lists  and  append  them. 

(APPEND  (MYMAPCAR  (FUNCTION  F00)  '(A  8  C))  (BAZ  Z  Z)) 

The  optimized  code  returned  by  die  transformations  was: 

(CONS  (FUNCALL  (FUNCTION  F00)  'A)  (MAPCAR  (FUNCTION  F00)  '(B  C ) ) ) 


Understanding  why  this  code  replaced  the  input  code  lakes  a  bit  of  study.  First  off,  the  programmer 
would  need  to  remind  himself  of  tltc  definition  of  MYMAPCAR  and  BAZ.  He  goes  to  his  file  and 
looks  them  up: 

(DEFUN  MYMAPCAR  (FUNARG  ARG) 

(CONO  ((NULL  ARG)  NIL) 

((ATOM  ARG)  (FUNCALL  FUNARG  ARG)) 

(T  (CONS  (FUNCALL  FUNARG  (CAR  ARG)) 

(MAPCAR  FUNARG  (CDR  ARG)))))) 


(DEFUN  BAZ  (X  Y) 

(COND  ((EQ  X  Y)  NIL) 

(T  (LIST  X  Y)))) 

To  figure  out  how  dtese  definitions  together  with  die  transformations  in  the  set  he  applied  worked 
together  to  form  the  replacement  code,  his  analysis  might  proceed  like  this: 

“Oh  no.  I’ve  been  QUUXHD!  What  happened  to  the  call  to  APPE  ND?  It's  turned  into  a  call  to 
CONS!  How  can  that  be? 

1  linmm.  Well,  procedure  integration  has  occurred.  I  guess.  The  definition  of  MYMAPCAR  has 
been  expanded  in-line,  but  since  the  second  argument  is  known  to  be  a  list,  the  tests  for  nullncss 
and  atomicity  were  eliminated,  leaving  only  die  CONS.  But  what  happened  to  my  BAZ  function?  I 
certainly  didn’t  give  it  any  constants,  so  what  makes  it  diink  it  can  go  away?  l  et's  sec. . .  If  I  expand 
its  definition  in-line,  I  get  a  conditional  whose  first  test  is  (EQ  Z  Z).  Oh,  of  course!  Iliat  must 
be  true,  so  it  returns  NIL.  Then  APPENO  of  somediing  and  NIL  leaves  that  something,  which 
explains  why  only  the  CONS  is  left.  Whew!” 

That  was  only  a  simple  example.  Imagine  trying  to  verify  output  from  a  transformation  system 
in  which  there  arc  several  levels  of  procedure  integration,  hairy  function  definitions,  obscure  test 
eliminations,  and  nested  lambda  expressions  that  produce  several  layers  of  bindings  one  can't  pos¬ 
sibly  remember  and  translate  between  all  at  once.  If  one  must  go  dirough  that  sort  of  analysis  for 
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each  piece  of  code  in  order  to  follow  the  activities  of  a  sourcc-to-sourcc  transformation  system, 
the  advantages  of  being  able  to  do  so  lose  some  of  their  appeal.  The  transformations  applied  to 
the  input  program  “do  violence”  to  the  user’s  code;  it  is  often  difficult  or  even  impossible  to  under¬ 
stand  why  a  transformation  did  or  did  not  apply,  and  how  the  various  transformations  intcjactcd  to 
produce  the  output  program. 

To  understand  what  occurred  during  the  transformation  process  requires  a  great  deal  of  effort 
on  the  part  of  the  user.  He  must  compare  the  input  source  with  the  output  source  to  sec  where 
changes  were  made.  He  must  look  up  function  definitions  to  sec  how  procedure  integration  took 
place.  To  understand  which  transformations  applied,  he  must  have  a  good  idea  of  the  transforma¬ 
tions  attempted  and  what  their  prerequisites  for  application  were.  It  may  be  impossible  for  him  to 
discover  what  transformations,  if  any,  almost  applied,  and  why  they  didn’t.  Furthermore,  without 
careful  study  of  the  system  itself,  he  has  no  clue  as  to  the  order  in  which  the  transformations 
applied,  and  how  this  may  have  affected  the  final  result. 

In  case  the  reader  is  not  convinced,  1  include  one  last  example  (without  exposition)  for  him  to 
ponder  over.  The  original  program  function.. 

(DEFUN  EXAMPLE  ( F00  BAR  BAZ) 

(COND  ((ATOM  FOO)  NIL) 

((ISINDEXED  FOO)  (PROCESS-INDEX  FOO  BAR  BAZ)) 

(  ...  (GET-FVJ'  FOO)  ...  ) 

(T  ...  ))) 


The  resulting  code: 

(DEFUN  EXAMPLE  (FOO  BAR  BAZ) 

(COND  ((ATOM  FOO)  NIL) 

((AND  (ATOM  (CAR  FOO)) 

(NUMBERP  (CAR  FOO))) 
(PROCESS-INDEX  FOO  BAR  BAZ)) 

(  ...  (CAR  FOO)  ...  ) 

(T  ...  ))) 


Relevant  function  definitions: 

(DEFUN  ISINDEXED  (X) 

(ANO  (NOT  (ATOM  X)) 

(ATOM  (CAR  X)) 

(NUMBERP  (CAR  X)))) 

(DEFUN  GET-PART  (Y) 

(COND  ((ATOM  Y)  NIL) 

((AND  (ATOM  (CAR  Y)) 

(NUMBERP  (CAR  Y) ) ) 
(CADR  Y)) 
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(T  (CAR  Y)))) 

With  some  study  and  comparison  of  the  two  definitions  of  EXAMPLE  given  here,  we  can  eventually 
see  that  they  arc  equivalent,  and  determine  what  transformations  were  applied  to  get  the  resulting 
code.  However,  this  can  be  a  tedious  process  for  die  entire  set  of  program  functions,  and  one  that 
could  be  automated.  I  propose  to  aid  die  user  in  understanding  the  transformations  performed 
above  (for  example)  by  having  available  to  him  information  about  die  changes  made,  lhc  user 
might  ask  where  the  (CAR  F00)  in  die  resulting  program  came  from,  and  die  system  would 
answer  that  it  came  from  the  simplification  of  (COND  (T  (CAR  F00) )),  which  was  the  result  of 
test  elimination  from  the  call  to  GET-PART  which  was  expanded  in-line. 
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JUST  WHAT  IS  an  “accountable”  transformation  system?  We  need  to  understand  what  is 
meant  by  the  term  and  how  the  characteristic  of  accountability  might  be  used  to  solve  the 
problems  presented  in  the  previous  chapter.  Then  we  may  design  and  build  such  a  system. 


2.1.  A  Definition 

An  accountable  sourcc-to-source  transformation  system  is  one  which  records  in  some  acces¬ 
sible  fashion  the  circumstances  of  application  of  any  transformations  it  performs,  lliat  is,  not 
only  docs  such  a  system  return  a  transformed  program,  but  it  leaves  behind  a  history  of  the  trans¬ 
formation  process  as  well.  Mechanisms  arc  provided  which  will  allow  the  user  to  easily  obtain 
information  from  this  history.  Thus,  a  programmer  might  well  receive  direct  answers  to  such 
questions  as: 

•  “What  initial  transformation  (if  any)  occurred  to  start  the  chain  of  transformations  performed 
on  this  section  of  code?” 

•  “What  did  this  piece  of  code  look  like  right  before  this  transformation  applied?” 

•  “What  would  this  piece  of  code  look  like  if  this  transformation  had  been  turned  off?” 

•  “Which  transformations  applied  at  this  point  in  the  program?  Why?” 

•  "What  transformations  did  not  apply?  Why  not?” 

•  "Which  transformations  always  applied?” 
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One  might  wonder  if  this  ability  is  possible  only  in  a  source- to- source  transformation  system, 
or  whether  optimizing  compilers  and  other  transformation  systems  could  be  made  accountable  as 
well.  The  point  of  an  accountable  system  is  that  it  aids  the  user  in  understanding  why  certain 
decisions  were  made,  where  they  were  made,  and  what  those  decisions  were.  If  W'C  assume  }hat  such 
information  is  useful,  it  is  natural  to  then  assume  drat  the  user  is  interested  in  studying  the  code  of 
the  resultant  program  as  well.  1  wish  to  make  no  claims  about  the  amount  of  interest  die  typical 
programmer  has  in  object  code,  but  1  will  admit  that  in  theory,  any  transformation  system  can  be 
made  accountable.  In  practice,  however,  a  sourcc-to-sourcc  transformation  system  seems  to  be  the 
best  candidate  for  such  an  improvement 


2.2.  Motivation  and  Uses  for  an  Accountable  System 

An  accountable  sourcc-to-sourcc  transformation  system  makes  all  the  advantages  of  a  transfor¬ 
mation  system  listed  in  Chapter  One  realistic.  The  output  of  an  accountable  system  is  not  only 
accessible  but  manageable,  and  the  transformation  process  is  rendered  truly  interactive. 


2.2.1.  Interactive  Transformation 

As  mentioned  in  the  section  on  source-to-source  transformation  systems,  although  such  sys¬ 
tems  provide  the  programmer  with  access  to  the  transformed  code,  for  him  to  understand  exactly 
what  took  place  requires  that  he  invest  a  significant  amount  of  time  and  effort.  He  must  not  only 
compare  input  and  output,  but  have  a  good  understanding  of  the  transformation  system  itself.  By 
automatically  recording  and  providing  access  to  the  information  Uiat  a  programmer  needs  to  decide 
if  the  transformation  process  accomplished  all  dial  he  desired,  we  supply  yet  another  tool  for  the 
program  development  process.  With  the  knowledge  that  such  a  tool  is  available,  a  programmer  is 
more  apt  to  concentrate  on  construction  and  verification  of  a  program  first  and  leave  the  problem 
of  efficiency  until  later.  When  its  time  comes,  die  user  is  able  with  the  use  of  an  accountable  system 
to  direct  the  transformation  process.  It  is  the  accumulation  of  such  tools  which  smooths  die  way  for 
development  of  automatic  program  synthesis  systems. 


2.2.2.  Debugging 

An  accountable  system  is  a  debugging  aid  in  the  sense  that  where  a  transformation  may  not 
have  had  the  intended  effect,  or  the  previous  application  of  some  other  transformation  caused  this 
one  to  fail  to  apply,  the  system  could  be  queried  as  to  why  a  given  transformation  did  not  apply,  or 
what  the  state  of  the  program  was  at  die  point  the  user  expected  die  transformation  to  apply.  The 
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answers  would  help  tire  programmer  to  locate  errors  in  the  transformation  set. 
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2.3.  Related  Work 

Other  systems  have  done  some  work  in  the  line  of  accountability  [Davis  1978],  but  most  of 
them  consist  of  little  more  than  a  simple  trace-back  of  the  computations  performed  in  the  program. 
A  full  or  even  partial  trace  of  what  happened  has  limited  usefulness,  however,  and  the  thought 
of  having  to  wade  through  one  is  enough  to  deter  all  but  the  most  desperate.  What  is  really 
needed  is  a  mechanism  whereby  the  available  information  about  a  computation  is  sifted  through 
and  sorted,  and  then  only  the  information  which  is  relevant  is  passed  on  to  die  user.  But  even  a 
fancy  trace  system  may  do  that  much.  I  know  of  no  extant  transformation  system  which  makes 
any  serious  attempt  at  understanding  its  own  process  and  accounting  for  that  process  to  the  user. 
Instead,  developers  of  current  transformation  systems  have  concentrated  their  efforts  on  ordering 
the  application  of  transformations,  proving  their  transformations  to  be  equivalence  preserving,  and 
experimenting  w  ith  various  types  of  control  for  the  systems  [l.ovcman  and  Faneuf  1975]  [Wcgbrcit 
1976]  [Gerhart  1975]  [Nicvcrgclt  1965]  [Schwartz  1974]  [Schciflcr  1977]. 


2.4.  Understanding  the  User’s  Needs 

What  information  docs  a  user  need  to  understand  the  transformation  process?  We  already 
have  some  idea  from  working  through  the  two  examples  in  Chapter  One.  Certainly  the  information 
needed  to  answer  the  questions  listed  at  the  beginning  of  this  chapter  would  be  useful.  Notice  that 
some  of  the  questions  asked  information  about  a  particular  piece  of  code,  while  others  requested 
information  about  a  transformation.  That  is,  one  might  ask  "What  arc  all  the  transformations  that 
applied  to  this  piece  of  code?”  or  "Where  arc  all  the  places  in  the  code  that  this  transformation 
applied?"  (or  “was  attempted”,  or  "didn’t  apply”).  The  types  of  questions  a  person  would  ask 
depend  on  that  person’s  goals  and  reasons  for  using  the  accountable  system  in  die  fust  place. 

I.ct  us  dicn  consider  possible  goals:  Someone  who  was  developing  the  transformational  com¬ 
ponent  of  this  system  might  wish  to  determine  the  effectiveness  of  a  particular  transformation  or 
be  interested  in  following  the  interaction  of  various  transformations.  That  person  would  tend  to 
have  more  questions  regarding  particular  transformations.  For  example,  die  type  of  information 
that  would  have  been  helpful  to  me  in  debugging  would  have  been  the  ability  to  ask  what  trans¬ 
formations  did  not  apply,  or  perhaps,  which  transformations  always  applied.  On  the  other  hand, 
a  programmer  using  diis  system  to  optimize  his  program  would  be  more  likely  to  ask  questions 
relative  to  specific  pieces  of  code.  For  example,  he  might  ask  if  a  certain  function  was  changed  at 
all,  or  why  a  certain  transformation  was  able  to  apply  to  the  piece  of  code  of  interest. 
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Hither  of  these  are  valid  approaches,  and  each  of  them  was  listed  as  a  reason  for  using  an 
accountable  system.  Thus,  we  will  attempt  to  record  information  which  can  be  used  to  answer 
questions  that  any  of  these  potential  users  might  have.  In  fact.  I  claim  that  the  same  information  is 
necessary  to  answer  any  of  them,  and  that  only  their  view  point  differs.  The  information  needed  is: 

(a)  that  a  transformation  was  attempted  at  some  point  in  die  code,  and  whether  or  not  it  success¬ 
fully  applied; 

(b)  what  any  piece  of  code  looked  like  following  any  transformation; 

(e)  tilings  known  to  be  true  (or  false)  at  any  given  location  in  code; 

(d)  how  the  information  in  (c)  came  to  be  known; 

(e)  for  each  individual  transformation:  what  must  be  true  to  enable  successful  application 
(prerequisites),  and  what  is  true  following  a  successful  application. 

From  this  information.  I  claim  we  can  derive  the  answers  to  any  of  the  questions  that  a  user 
might  ask  (which  have  been  suggested  so  far). 


2.5.  The  Design 

I  believe  that  any  mechanisms  for  recording  what  goes  on  during  the  transformation  process 
should  be  independent  of  those  mechanisms  which  perform  the  transformations.  That  is.  an  ac¬ 
countable  transformation  system  can  be  viewed  as  two  separate  processes:  one  behaves  just  as  cur¬ 
rent  transformation  systems  do  in  that  it  decides  when  and  where  to  perform  die  transformations, 
thm  does  so;  the  other  process  stands  by  and  quietly  watches,  taking  note  of  all  such  decisions.  The 
advantage  of  this  approach  is  that  already  extant  transformation  systems  can  be  made  accountable 
with  less  hassle,  and  furthermore,  the  division  ol  labor  simplifies  the  task  of  constructing  an  ac¬ 
countable  system.  I  will  occasionally  refer  to  the  lirst  process  as  the  "transformational  component”, 
and  to  the  second  as  the  “accountable  component".  As  we  will  see  in  more  detail  later  on,  the 
accountable  component  will  be  further  divided  into  two  sub  components:  a  recording  element  and 
a  query  element. 

The  two  processes  viewed  together  appear  to  the  user  as  a  transformation  wstem  that  is  aware 
of  its  own  actions,  and  which  can  then  report  to  the  user  what  it  did,  step  by  step.  An  interesting 
variation  of  ibis  approach  might  be  a  system  that  reported  what  it  did  as  it  did  it.  rather  than 
waiting  until  the  end  to  provide  the  user  with  information.  Although  this  method  has  sonic  merit 
and  deserves  to  be  studied,  its  one  dear  disadvantage  is  that  it  gives  the  user  less  choice  about  what 
information  to  receive.  Though  he  might  have  the  option  before  the  process  began  to  specify  what 
types  of  decisions  to  be  told  about,  it  would  be  dillicult  for  him  to  ask  for  other  information  based 
on  what  he  raw  during  the  process.  Furthermore,  if  the  program  to  be  transformed  is  very  large, 
or  the  accountable  system  is  very  slow,  the  user  may  not  wish  to  "watt  around"  for  the  information 
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he  requested  to  be  displayed.  Instead,  while  the  transformation  system  is  cooking,  die  programmer 
can  go  off  and  have  a  cup  of  tea.  Upon  their  mutual  return,  he  may  ask  for  whatever  information 
he  desires,  decide  on  the  basis  of  that  information  what  else  he  wishes  to  see,  dien  immediately 
study  that  information  as  well.  One  reason  to  favor  having  a  “real-time”  report  would  be  .if  on  die 
basis  of  information  reported,  the  user  could  then  opt  to  change  the  course  of  the  transformation 
process.  This  feature  is  beyond  die  scope  of  our  discussion  of  an  accountable  system,  though  I  will 
venture  to  suggest  diat  such  decisions  perhaps  should  be  automated;  dial  is,  dicy  should  be  part  of 
the  transformation  system  itself. 

For  a  system  to  account  for  its  actions  (and  those  actions  arc  the  transformations  it  docs  or 
docs  not  successfully  perform),  it  must  keep  some  sort  of  record  of  what  it  is  doing  as  it  executes, 
and  be  able  to  later  access  that  record  in  a  meaningful  way.  We  must  diereforc 

(1)  collect  and  record  the  relevant  information  in  some  reasonable  fashion,  and 

(2)  access,  process,  and  return  that  information  to  the  user. 

These  tasks  will  be  performed  respectively  by  the  recording  and  query  elements  of  the  accountable 
component. 


2.5.1.  The  Recording  Element 

Let’s  again  go  over  die  information  we  decided  in  section  2.4  to  collect: 

(a)  We  must  record  that  a  transformation  was  attempted  at  some  point  in  the  code,  and  whether  or 
not  it  successfully  applied. 

The  time  to  collect  this  information  is  clearly  at  the  time  of  transformation  application.  1 
have  not  yet  discussed  exactly  what  constitutes  a  transformation,  nor  what  its  inputs  and  out¬ 
puts  arc.  It  seems  reasonable  to  suggest,  however,  that  one  thing  a  transformation  migh'  K 
responsible  for  returning  is  a  flag  as  to  whether  or  not  it  applied.  This  information  can  then  \  j 
recorded  by  die  recording  element. 

(b)  We  must  record  what  any  piece  of  code  looked  like  following  any  transformation. 

This  simply  means  we  must  keep  track  of  all  the  intermediate  results  of  transformation.  We 
can't  "let  go"  of  an  old  expression  when  a  transformation  applies  which  replaces  that  expres¬ 
sion  with  some  equivalent  one.  Hie  time  to  update  this  information  is,  again,  at  the  time  of 
transformation  application,  but  only  when  a  transformation  applies  successfully ,  since  only  at 
such  times  will  expressions  change.  We  will  assume  that  each  successful  transformation  returns 
not  only  a  flag  signifying  that  it  applied,  but  the  results  of  its  application  as  well. 

(c)  We  must  record  things  known  to  be  true  (or  false)  at  any  given  location  in  code. 

By  this  I  mean  information  relating  to  context.  For  example,  code  which  is  executed  follow¬ 
ing  the  true  branch  of  a  test  for  x  =  0  may  assume  diat  x  =  0  is  true.  Recording  this  information 
is  complicated  by  the  fact  diat,  for  any  given  point  in  code  (and  there  arc  many!),  what  is  true 
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at  one  time  may  not  be  true  at  another,  because  the  results  of  transformation  application  may 
change  the  context  of  an  expression.  We  will  discuss  this  issue  in  more  detail  in  Chapter  ITirce. 
For  now,  suffice  it  to  say  that  there  will  be  an  initial  context  to  record,  and  then  some  updating 
necessary  as  each  transformation  applies. 

(d)  We  must  record  how  the  information  in  (c)  came  to  be  known. 

This  consilutcs  a  justification.  Let  us  say  some  transformation  t  applies  to  a  piece  of  code. 
Its  application  depends  on  some  truth  p.  Although  upon  being  queried  as  to  why  t  applied, 
die  system  might  return  p,  we  might  also  wish  to  understand  why  p  is  true.  Thus  each  truth 
must  also  have  a  justification.  Recalling  the  example  given  above  in  which  x  =  0  was  tme,  a 
transformation  using  that  information  would  be  responsible  for  reporting  dial  it  did.  If  we  dicn 
needed  to  he  reminded  why  x  =  0  was  true,  we  should  be  able  somehow  to  obtain  the  reply 
"This  code  is  part  of  the  true  branch  following  a  test  for  x=0”,  and 
then  perhaps  even  a  pointer  to  diat  test.  The  time  to  collect  this  information  is  at  the  time  that 
pis  recorded.  T  hat  is,  every  truth  must  carry  with  it  a  justification;  one  can  not  be  recorded 
w  ithout  the  other. 

(e)  For  each  individual  transformation  we  must  record  (1)  what  must  be  true  to  enable  successful 
application  (prerequisites),  and  (2)  what  is  true  following  a  successful  application. 

T  his  is  static  information  and  only  needs  to  be  recorded  once.  It  is  associated  with  a  transfor¬ 
mation  (as  opposed  to  a  section  of  code),  and  should  be  accessible  given  only  the  name  of  a 
transformation.  It  is  in  some  sense  part  of  the  description  of  a  tranformation,  though  discussion 
of  how  the  information  is  represented  will  be  postponed  until  Chapters  Three  and  Four. 

We  may  now  summari/.c  die  task  of  the  recording  element  of  the  accountable  system. 
Transformations  must  include  in  their  description  a  set  of  input  and  output  assertions,  available 
upon  request,  before  any  transformations  apply,  a  set  of  truths  for  each  piece  of  code  (dependent 
on  its  context)  must  be  collected  and  recorded  along  with  their  justifications.  After  each  transfor¬ 
mation  is  attempted,  it  must  provide  to  the  recording  element  die  following  information: 

(1 )  w  hat  code  it  attempted  to  transform; 

(2)  w  hether  or  not  it  was  successful; 

(3)  if  it  was  successful,  the  resultant  code. 

T  he  recording  element  must  represent  this  information  in  an  efficient  and  accessible  manner. 

Notice  that,  in  all  our  discussion,  no  mention  has  been  made  of  how  a  transformation  comes  to 
apply.  Obviously  one  succeeds  if  and  only  if  all  of  its  input  prerequisites  arc  met,  but  even  before 
that  can  occur  a  transformation  must  be  attempted  in  the  first  place.  T  he  mechanism  which  controls 
this  process  is  independent  not  only  of  the  recording  element,  but  of  the  accountable  component 
as  a  whole.  It  constitutes  the  ^informational  component  of  the  system,  and  need  not  be  aware  at 
all  that  its  actions  arc  being  monitored.  Similarly,  recall  that  I  consider  the  accountable  component 
merely  a  passive  observer  of  the  transformation  process  implemented  by  the  transformational  com- 
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ponent;  it  is  not  concerned  with  nor  responsible  for  decisions  made  by  that  component.  This  is  not 
the  contradiction  it  appears  to  be.  For  although,  when  viewed  as  two  components,  one  works  and  the 
other  only  blindly  records,  when  they  are  viewed  as  one  system,  as  far  as  the  user  is  concerned  that 
system  accounts  for  its  actions. 


2.5.2.  The  Query  (element 

As  a  result  of  recording  all  of  the  above  information,  we  have  die  ability  to  provide  not  just 
a  fancy  trace  of  all  the  transformations  performed,  but  an  historical  “slice”  of  information  about 
any  subtree  of  the  program.  That  is,  if  we  imagine  the  set  of  intermediate  program  trees  which 
represent  the  entire  program  after  each  transformation  has  applied,  the  accountable  component 
(via  the  query  element)  can  "cut  across  time"  (so  to  speak)  and  return  only  the  information  relevant 
to  changes  made  in  a  particular  subtree  of  the  program.  For  example,  suppose  a  programmer 
expected  a  certain  function  call  to  be  replaced  by  its  argument.  He  should  be  able  to  somehow 
"point"  to  the  spot  in  code  and  inquire  as  to  whether  this  substitution  occurred  and  why  or  why 
not. 

From  grammar  school  days,  we  remember  that  all  complete  sentences  must  have  both  a  sub¬ 
ject  and  a  verb.  The  subject  may  be  implied,  as  in  the  command  “Go”  which  expands  to  “You  go". 
Or  die  verb  may  be  implied,  as  “l"  in  answer  to  the  question  “Who  did  it?",  which  expands  to  “I 
did  it".  Similarly,  wdien  one  asks  a  question,  both  a  subject  and  a  verb  must  be  present,  expressed, 
or  implied.  In  the  query  element,  we  need  a  mechanism  for  "pointing”  to  the  subject  of  our  query, 
and  a  mechanism  for  specifying  the  action  or  verb  we  wish  to  invoke. 


2.5.2.I.  Pointing  to  the  Subject 

If  the  subject  is  a  transformation,  we  may  refer  to  that  transformation  by  its  name.  A  transfor¬ 
mation  never  changes  (if  it  does,  it  is  no  longer  the  same  transformation)  and  is  therefore  easy  to 
point  to. 

Hut  now  consider  die  problem  if  the  subject  is  a  piece  of  code.  We  have  no  easy  way  to  point 
to  it,  unless  the  code  is  a  function  definition,  in  which  ease  we  can  refer  to  it  by  name.  To  point  to 
a  piece  of  a  function  definition,  we  must  describe  Uiat  piece:  “The  first  predicate  of  the  conditional 
clause  which  is  the  body  of  die  function  definition  F00"  or  “  The  second  argument  to  the  function 
call  BAR  where  that  call  occurs  in  the  call  to  BAZ  in  the  value  returned  from  the  second  clause 
of  die  conditional  expression  which  is  bound  to  die  dummy  variable  TREE  in  die  LET  statement 
which  forms  die  body  of  the  function  definition  of  QUUX.”  That  sounds  pretty  awful;  would  you 
like  to  see  it? 

(DEFUN  QUUX  (A  B  C) 
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(LET  ((TREE  (CONO  ((NULL  A)  NIL) 

(T  (BAZ  (BAR  B  (CAR  A))))))) 

(CONO  ((NULL  TREE)  (LIST  A  B  C)) 

(T  (CONS  C  TREE))))) 

That's  really  not  so  bad,  is  it?  the  problem  is  that  even  to  point  to  the  code  ( CAR  A )  in  the  trivia) 
definition  above  requires  a  lot  of  blabbering.  Another  method  might  be  simply  to  say  "The  first  (or 
hth’)  occurrence  of  die  call  ( CAR  A )  in  the  definition  of  QUUX-'.  but  there  are  cases  when  having 
to  count  the  occurrences  of  the  desired  expression  (or  even  giving  the  desired  expression)  would  be 
more  complicated  than  referring  to  the  structure  of  the  context.  For  example,  consider  a  lengthy 
function  definition  in  which  you  wished  to  point  to  a  certain  tondilion.il  expression.  There  very 
well  might  be  a  dozen  conditional  expressions  in  the  definition,  and  you  would  have  to  look  for  and 
find  diem  all  until  you  came  to  the  one  you  wanted  in  order  to  know  die  value  of  n.  Not  that  the 
first  way  is  easier:  both  methods  arc  painful.  A  simple  solution  to  die  problem  of  pointing  would  be 
to  use  a  graphics  terminal  equipped  with  a  mouse;  however,  I  don't  want  this  system  to  have  to  rely 
on  those. 

Another  complicating  factor  arises  if  we  decide  to  ask  about  some  piece  of  code  diat  only 
occurs  in  an  intermediate  version  of  the  program.  That  means  diat  to  specify  the  code  we  wish  to 
use  as  the  subject  of  our  query,  we  must  not  only  describe  the  context  of  the  parse  tree  it  occurs 
in.  but  specify  die  time  at  which  that  tree  occurred.  What  constitutes  time  in  our  transformation 
system? 

Since  the  passage  of  time  around  some  object  can  only  be  detected  by  the  observation  of 
change  in  that  object,  wc  must  consider  what  might  bring  forth  changes  in  the  argument  program. 
The  successful  application  of  a  uansformalion  certainly  docs,  though  an  unsuccessful  attempt  docs 
not  change  the  argument  program.  Assuming  that  the  system  w  ill  operate  on  a  sequential  machine 
(as  opposed  to  the  complexities  involved  in  dealing  with  parallel  processes  or  parallel  machines), 
then  defining  a  unit  of  time  to  be  the  application  of  a  transformation  seems  appropriate,  l^atcr 
wc  will  see  whether  this  definition  is  appropriate  when  dealing  with  unsuccessful  transformation 
attempts. 

We  can  now  label  each  unit  of  time.  The  input  program  is  labelled  version  0,  and  if  there  were 
n  successful  transformation  applications,  then  the  resultant  output  program  is  version  n.  For  any  k 
such  that  0  <  k  <  n,  version  k  is  the  program  produced  by  the  first  k  transformation  applications. 
Or,  said  another  way,  version  k  of  die  program  is  the  result  of  applying  the  fcth  transformation  to 
version  k  —  l  of  the  program. 

This  description  of  time  is  useful  not  only  in  the  the  query  clement,  but  in  the  recording 
element  as  well.  That  element  will  need  a  means  of  distinguishing  the  record  of  one  version  from 
die  record  of  another. 

But  now  back  to  the  problem  at  hand:  pointing  to  a  particular  piece  of  code.  Wc  know  we 
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can  specify  the  version  of  the  tree  we  wish  used  as  a  search  domain  by  giving  the  correct  numerical 
label.  That  can  be  a  big  problem  if  we  don't  know  at  what  timc(s)  the  tree  was  extant! 

To  limit  die  possibilities,  let's  constrain  them.  First,  we  will  assume  that  the  programmer  is 
most  familiar  with  the  version  of  die  program  he  wrote.  Thus,  if  at  any  time  he  wishes  to  ask  about 
a  particular  piece  of  code,  most  likely  he  mentally  sees  llic  subject  code  as  it  occurs  in  version  0  of 
the  program  and  is  able  to  describe  the  location  of  the  code  he  wishes  to  ask  about  as  it  appears 
in  that  version,  lltcrefore  we  state  that  the  initial  search  for  a  piece  of  code  to  point  at  will  be 
made  using  version  0  of  the  program.  Secondly,  given  the  tree  to  use  as  a  search  domain,  in  order 
to  specify  that  subtree  of  it  to  which  one  wishes  to  point,  one  first  specifies  the  relevant  function 
definition  name  (QUUX  in  die  example  above).  Then  the  programmer  must  somehow  direct  the 
query  cement  to  the  desired  piece.  Since  1  -ISP  lends  itself  so  readily  to  the  use  of  tree  structures,  1 
have  decided  (for  now)  that  the  most  painless  way  of  referring  to  a  piece  of  code  is  to  say  “down”, 
"over”,  or  “up"  until  die  correct  piece  is  pointed  at.  Thus,  to  point  at  (CAR  A)  in  the  example 
above.  1  would  give  the  function  name  QUUX.  then  the  string  of  commands  “down,  over,  over, 
over  (now  it’s  pointing  to  the  LET  statement),  down,  over,  down,  down,  over  (now  it’s  pointing 
to  the  first  conditional  expression),  down,  over,  over,  down,  over  (now  it’s  pointing  to  the  call  to 
BAZ).  down,  over,  down,  over,  over  (and  now  it's  pointing  to  die  call  (CAR  A)).”  Now  that  also 
sounds  painful,  but  if  a  cursor  responds  immediately  to  each  command  by  jumping  in  front  of  tire 
currently  pointed  to  expression  as  displayed  on  the  terminal  screen,  then  it’s  not  so  bad.  Once  wc 
discuss  the  implementation  of  our  system,  things  may  get  easier,  but  for  now  wc  will  settle  for  this 
method  of  code  specification. 


2.5.2.2.  Specifying  the  Action 

Specifying  the  action  to  perform  or  some  piece  of  information  to  return  about  the  subject  is 
considerably  easier  than  pointing  to  the  code.  If  wc  imagine  a  menu  of  available  information,  all 
one  must  do  is  select  an  item  from  that  menu,  and  die  desired  dish  of  information  should  be  served 
immediately.  Designing  the  menu  to  meet  the  needs  of  the  user  is,  however,  an  important  and  non¬ 
trivial  task.  I  have  discussed  already  (in  section  2.4)  the  information  wc  have  decided  to  provide  the 
user.  Let's  now  categorize  this  information  further  into  some  sort  of  draft  menu.  Whether  or  not 
providing  this  information  is  actually  feasible  will  be  discussed  in  Chapter  Four. 

Given  a  transformation  as  the  subject  of  a  query,  wc  can  ask  the  following: 

•  (n)  number  of  times  attempted  —  Returns  a  number  which  represents  die  number  of  times  diis 
transformation  was  attempted. 

•  (+)  positive  attempts  —  Returns  a  number  which  represents  the  number  of  times  dais  transfor¬ 
mation  successfully  applied. 

•  (-)  negative  attempts  —  Returns  a  number  which  represents  the  number  of  times  diis  transfer- 
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ination  failed  to  apply. 

•  (<a  ntimbcr>)  some  number  n  —  Prompts  for  either  a  +  or  a  then  returns  the  piece  of  code 
which  was  the  argument  to  the  transformation  the  nth  time  it  succeeded  or  failed,  respectively. 

•  ( i )  input  assertions  —  Returns  those  truths  which  arc  prerequisites  for  application  of  this  trans¬ 
formation. 

•  (o)  output  assertions  —  Returns  those  truths  which  will  be  changed  as  a  consequence  of  the 
successful  application  of  this  transformation. 

Given  a  piece  of  code  as  the  subject  of  a  query,  we  may  ask  the  following  questions: 

•  (+)  next  change  to  subject  tree  —  Returns  the  piece  of  code  as  it  appears  following  die  transfor¬ 
mation  which  next  applied  to  it  at  die  current  level  (as  opposed  to  one  of  its  subtrees).  If  no 
transformation  applied  at  diat  level,  it  says  so  and  returns  the  same  piece  of  code. 

•  (-)  previous  change  to  subject  tree  —  Returns  die  piece  of  code  as  it  appears  previous  to  the 
transformation  which  last  applied  to  it  at  the  current  level.  If  no  transformation  has  applied  at 
that  level,  it  says  so  and  returns  die  same  piece  of  code. 

•  (n)  next  version  of  subject  tree  —  Returns  die  same  tree  as  it  appears  following  the  transforma¬ 
tion  which  next  applied  to  any  subtree  of  the  subject  tree.  If  dicrc  is  no  change,  it  says  so  and 
returns  die  same  piece  of  code. 

•  (p)  previous  version  of  subject  tree  —  Returns  the  same  tree  as  it  appears  previous  to  die  trans¬ 
formation  which  last  applied  to  any  subtree  of  the  subject  tree.  If  dicrc  was  no  change,  it  says  so 
and  returns  die  same  piece  of  code. 

•  (1 )  last  version  of  subject  tree  —  Returns  die  final  version  of  the  subject  tree,  as  it  appears  in  the 
output  of  the  transformational  component. 

•  (v)  current  version  of  subject  tree  —  Returns  a  number. 

•  (x)  transformation  which  created  current  version  of  subject  tree  —  Returns  die  name  of  the 
transformation. 

•  (t)  truths  known  —  Returns  a  list  of  the  tilings  known  to  be  true  for  subject  tree  at  the  current 
point  in  time,  and  for  each  truth,  its  justification. 

•  (s)  set  version  —  Prompts  for  some  number  n  and  returns  the  subject  tree  as  it  appears  in 
version  n. 


2.S.2.3.  An  Example 

Given  this  menu  of  information,  let’s  dredge  up  die  function  call  used  back  in  Chapter  One 
and  see  if  our  new  accountable  component  with  its  recording  and  query  elements  gets  us  anywhere. 
Our  input  to  die  transformation  system  (used  with  some  canned  set  of  optimizing  transformations) 
is  die  following  function  call: 
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(APPEND  (MYMAPCAR  (FUNCTION  FOO)  '(A  B  C))  (BAZ  Z  Z)) 

For  now  wc  will  assume  that  the  system  already  knows  the  definitions  of  MYMAPCAR  and  BAZ.  We 
set  the  query  element  to  point  to  version  0  of  llie  call  to  APPEND.  Now  what?  We  can  ask  to  see 
what  this  tree  looked  like  at  any  point  in  time  (version  0  through  version  n  where  there  were  n 
total  transformations  applied).  The  system  when  it  is  through  transforming  will  tell  us  how  many 
transformations  applied,  that  is,  the  ‘■time”  at  the  end  of  the  run,  if  we  enter  the  ‘‘1”  command. 
Let’s  say  there  were  10  applications.  Then  we  ask  to  sec  version  10.  live  fully  optimized  code 
returned  by  the  system  looks  like: 

(CONS  (MYFUNCALL  (FUNCTION  FOO)  'A) 

(MYMAPCAR  (FUNCTION  FOO)  '(B  C))) 

Now  we  start  scratching  our  respective  heads.  First,  wc  arc  curious  to  learn  where  the  CONS  came 
from,  so  wc  enter  to  sec  what  was  here  before. 

(APPEND  (CONS  (MYFUNCALL  (FUNCTION  FOO)  'A) 

(MYMAPCAR  (FUNCTION  FOO)  '(B  C) ) ) 

NIL) 

"Ahlih...”  wc  say,  "the  call  to  APPEND  was  simplified:  it's  second  argument  was  NIL,  so  the  first 
argument  was  returned.”  And  in  fact,  wc  can  get  just  that  answer  if  wc  ask  for  the  name  of  the 
transformation  that  applied,  and  then,  using  that  as  our  subject,  inquire  as  to  its  input  assertions. 

We’re  pretty  sure  wc  understand  how  the  first  argument  to  APPEND  came  to  be,  but  wc  can’t 
figure  out  the  NIL.  So  wc  give  the  correct  combination  of  “downs”  and  “overs”  to  arrive  at  the 
NIL,  which  in  this  case  would  be  “down,  over,  over." 

NIL 

Now  wc  enter  to  learn  what  this  tree  was  before  it  was  NIL. 

(COND  (T  NIL)) 

That  wasn’t  incredibly  helpful;  let's  try  entering  again. 

(COND  (T  NIL) 

(T  (LIST  Z  Z))) 

The  code  begins  to  look  familiar,  but  obviously  what  wc  want  is  to  discover  how  it  comes  to  believe 
the  first  test  will  always  return  T.  Let’s  "down,  over,  down’’  our  way  to  the  first  predicate  and  enter 
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Now  it's  all  clear!  Since  we  called  BAZ  with  die  arguments  Z  and  Z,  the  first  test  in  its  definition 
will  always  be  true.  But  let’s  say  we  didn’t  remember  the  call  to  BAZ  (maybe  you  don't!).  Wc  can 
back  up  to  the  conditional  expression  (“up,  up”)  and  ask  to  see  what  it  was  in  version  0. 

(BAZ  Z  Z) 

Now  wc  enter  “n”  to  sec  what  happened  to  this  tree  next. 

((LAMBDA  (X-0  Y-0) 

(COND  ( ( EQ  X-0  Y-0)  NIL) 

(T  (LIST  X-0  Y-0)))) 

Z  Z) 

If  wc  ask  for  the  transformation  which  produced  that,  it  would  say  Procedure  integration. 
Wc  can  live  with  that.  1'hc  strange  dummy  argument  names  were  created  (that’s  “gensymed”  for 
I  ISP  hackers)  to  avoid  variable  name  conflicts.  So  now  what?  If  the  programmer  still  doesn’t 
understand,  he  can  continue  to  ask  for  information  until  he  docs.  Wc  can  sec  at  this  point,  however, 
that  when  die  actual  arguments  are  substituted  for  die  dummy  argumcnLs  (Z  for  X-0,  and  Z  for 
Y-0),  dial  the  equality  test  will  change  to  (EQ  Z  Z),  which  explains  die  NIL  wc  had  way  back 
up  there. 

Although  I  wish  to  concentrate  on  the  issues  involved  in  implementing  an  accountable  source- 
to-sourcc  transformation  system,  wc  must  first  discuss  die  transformation  system  it  will  work  with. 
As  you  may  recall  from  section  2.5,  the  transformational  component  and  the  accountable  com¬ 
ponent  arc  to  be  independent  of  each  other;  the  transformational  component  is  not  aware  of  the 
existence  of  the  accountable  component,  and  though  the  accountable  component  is  aware  of  the 
transformational  component,  it  is  not  responsible  for  die  decisions  that  die  transformational  com¬ 
ponent  makes.  If  I  were  interested  in  discussing  issues  of  die  implementation  of  transformation 
systems,  then  I  could  ignore  the  issue  of  accountability  (and  indeed,  so  far  everyone  else  has 
ignored  it).  However,  since  the  accountable  component  both  observes  the  actions  of  the  other 
and  expects  certain  information  to  be  made  available  by  the  transformations,  before  describing  its 
implementation  I  will  first  discuss  the  transformational  component  of  the  accountable  system. 
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Tl  ik  transformational  COMPONENT  of  the  system  will  be  implemented  as  simply  and 
straightforwardly  as  possible  so  that  we  may  concentrate  on  the  more  interesting  issue  of 
accountability.  Though  sourec-to-source  transformation  systems  conceptually  operate  on  the  text 
of  a  high-level  language  source  program,  internally  they  typically  manipulate  a  non-tcxtual  repre¬ 
sentation,  similar  to  (or  often  identical  to)  a  parse  tree  of  the  program.  The  use  of  this  internal  form 
makes  the  system’s  access  to  pieces  of  the  structure  more  efficient  (as  opposed  to  having  to  re-parse 
the  program  every  time  the  system  needs  to  refer  to  a  part  of  it).  In  order  to  avoid  having  to  deal 
with  parsing  and  internal  representation,  I  made  several  decisions.  First,  I  chose  to  transform  LISP 
programs  only.  Since  die  LISP  parser  is  made  available  to  the  user  via  READ.  1  didn’t  have  to  write 
my  own.  Second,  1  chose  to  implement  the  transformation  system  in  LISP.  And  third,  I  decided  to 
hide  decision  number  one  from  decision  number  two.  Which  leads  us  to  the  subject  of  information 
hiding. 


3.1.  Information  Hiding 

Information  hiding  is  the  separation  of  data  definition  from  program  definition.  In  the  case 
of  the  transformation  system,  it  means  not  letting  the  implementation  realize  that  it  is  transforming 
LISP  code.  Instead,  whenever  it  wishes  u>  access  a  piece  of  data,  rather  titan  obtaining  that  piece 
directly  it  calls  a  procedure  to  do  it.  Iliat  is.  there  are  specific  data  selection  and  data  constructing 
procedures  which  are  called  whenever  data  is  manipulated.  For  example,  suppose  dial  we  have 
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( DE  FUN  FCAILIFUNC  (FUNCTION-CALL) 

(CAR  FUNCTION-CALL)) 

(OEFUN  F CALL ! ARGL 1ST  (FUNCTION-CALL) 

(CDR  FUNCTION-CALL)) 

(OEFUN  FCALL ! FIRST-ARG  (FUNCTION-CALL) 

(ARGSIARG  ( F CALL ! ARGL 1ST  F UNCT ION-CALL ) ) ) 

(OEFUN  FCALL ! SECOND- ARG  (FUNCTION-CALL) 

(ARGSIARG  ( ARGS ! RE S I ARGS  ( FCALL ! ARGL 1ST  FUNCTION-CALL)))) 

Table  3-1.  Selector  functions  for  the  Function  Call  structure  type. 


a  function  call  which  is  bound  to  the  atom  FCALL,  and  that  wc  wish  to  select  its  first  argumenL 
In  LISP,  a  function  call  is  represented  as  a  list,  with  the  operator  being  the  first  clement  and  the 
operands  following.  Thus,  the  first  argument  of  a  function  call  would  be  the  CADR  of  the  list. 
However,  instead  of  writing  (CADR  FCALL)  to  obtain  the  first  argument  though,  we  call  the 
procedure  FCALL !  FIRST-ARG,  defined  as: 

(DEFUN  FCALL! FIRST-ARG  (FCALL) 

(CADR  FCALL)) 

If  the  representation  of  a  function  call  ever  changes,  wc  only  need  to  rewrite  die  relevant  data 
selecting  and  constructing  functions.  The  program  itself  should  remain  unaffected.  Wc  can  think 
of  a  function  call  as  a  particular  data  type,  and  require  that  no  direct  reference  be  made  to  any 
instance  of  that  type  except  via  specific  operations  defined  for  that  type  only.  Similarly,  we  define 
a  list  of  arguments  to  be  a  separate  type  also,  hiding  the  representation  of  argument  lists  from 
the  system.  Wc  write  the  function  FCALL !  ARGLIST  which  takes  a  function  call  and  returns  the 
list  of  arguments,  and  the  function  ARGSIARG,  which  takes  an  argument  list  and  returns  the  first 
argument.  T  he  definition  of  FCALL !  FIRST-ARG  which  uses  these  two  new  functions  is  given  in 
Table  3-1.  In  addition,  wc  can  define  predicates  on  these  data  types  which  test  their  state.  For 
example,  by  calling  die  function  ARGS !  NULL?,  we  can  ask  whether  a  list  of  arguments  is  empty  or 
not.  An  extended  version  of  ARGS !  ARG  (show  n  in  Table  3-2)  uses  this  predicate  to  make  sure  that 
no  argument  list  is  empty. 

The  advantages  of  information  hiding  are  that  it: 

•  allows'greatcr  flexibility  in  choice  of  data  representation, 

•  enhances  self-documentation, 

•  encourages  one  to  consider  the  data  at  an  abstract  level,  apart  from  program  specification,  and 

•  facilitates  proving  program  correctness. 
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;;;  Selectors 

(DEFUN  ARGSIARG  ( ARGLIST) 

(CONO  ((ARGSINULL?  ARGLIST) 

(ERROR  ' | Argl ist  is  null  —  ARGS ! ARG |  ARGLIST)) 

(T  (CAR  ARGLIST)))) 

(DEFUN  ARGS ! RESTARGS  (ARGLIST) 

(COND  ((ARGSINULL?  ARGLIST) 

(ERROR  ' ( Argl  ist  is  null  —  ARGS!RESTARGS|  ARGLIST)) 

(T  (CDR  ARGLIST)))) 

; ; ;  Predicates 

(DEFUN  ARGSINULL?  (ARGLIST)  (NULL  ARGLIST)) 

;;;  Constructors 

(DEFUN  ARGS IADD-ARG  (ARG  ARGLIST)  (CONS  ARG  ARGLIST)) 

(DEFUN  ARGSIADD-ARGS  (ARGS  ARGLIST)  (APPEND  ARGS  ARGLIST)) 

Table  3-2.  Operations  for  the  Argument  List  structure  type. 


The  disadvantage  of  this  strategy  is  that  there  is  extensive  overhead  in  the  additional 
procedure  calls,  causing  execution  time  to  shoot  way  up  (by  about  300  per  cent  in  one  of  my 
programs).  However,  this  can  be  overcome  through  use  of  a  set  of  optimizing  transformations. 
Transformations  can  be  written  which  search  for  calls  to  the  selector  and  constructor  functions  and 
replace  these  calls  by  the  appropriate  code.  We  can  have  our  tea  and  drink  it  too.  Jn  addition,  trans¬ 
formations  can  be  written  to  eliminate  duplicate  tests  that  may  arise  as  a  result  of  such  procedure 
integration.  What  if  the  call  ( FCALL!  F IRST-ARG  FCALL)  was  made  at  a  point  in  code  where 
it  was  already  known  that  the  argument  list  of  FCALL  was  not  empty?  The  call  to  ARGS !  NULL? 
in  the  body  of  the  FCALL !  FIRST-ARG  would  be  redundant;  a  set  of  optimizing  transformations 
could  realize  this  and  eliminate  the  redundant  test.  Clearly,  information  hiding  (data  abstraction) 
and  program  transformation  go  hand  in  hand.  For  more  of  my  views  on  the  subject  of  information 
hiding,  sec  [Kerns  1977)  and  [Steele  1980). 


3.2.  Internal  Form 

Hy  using  this  information  hiding  technique.  1  have  essentially  hidden  the  internal  form  of  the 
argument  program  from  Lite  transformation  system.  This  will  be  important  later,  when  1  augment 
the  internal  form  of  the  argument  program,  using  it  to  store  more  than  just  pieces  of  program  text. 
At  that  point  having  special  functions  to  access  pieces  of  it  w  ill  simplify  the  task.  For  now.  however. 
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it  is  safe  to  tliink  of  the  internal  form  of  the  LISP  program  as  the  LISP  program  itself.  Some  sample 
accessing  and  constructing  functions  are  shown  in  fable  3-1  and  Table  3-2. 


3.3.  Control 

The  “control”  of  die  transformational  component  is  diat  process  which  models  a  human  pro¬ 
grammer  attempting  to  optimize  his  code.  He  may  have  a  catalo-  of  transformations  at  his  side, 
but  without  the  knowledge  of  how  to  apply  them  they  do  him  no  good.  He  must  recognize  that 
a  transformation  is  applicable  to  his  code,  understand  the  possible  interactions  between  multiple 
transformations  to  a  single  piece  of  code,  and  be  able  to  verity  that  a  transformation  is  valid.  Two 
of  the  most  difficult  problems  in  automating  this  process  arc  (1)  transformation  ordering  and  (2) 
information  gathering  (I.ovcman  1977).  By  “transformation  ordering,"  1  mean  die  decision  of  when 
to  apply  which  transformation  where.  ’’Information  gathering”  refers  to  the  ascertaining  of  various 
truths  about  pieces  of  code. 


3.3.1.  Transformation  Ordering 

Given  die  parse  tree  of  an  argument  program  and  a  set  of  transformations  to  apply  to  that  pro¬ 
gram,  die  order  in  which  those  transformations  arc  applied  may  determine  the  resultant  program 
tree.  For  example,  suppose  I  have  the  following  set  of  two  transformations: 

(1)  (APPEND  <X>  (APPEND  <Y>  <Z> ) )  =>  (APPEND  <X>  <Y>  <Z>) 

(2)  (APPEND  NIL  <X>)  =>  <X> 

1  wish  to  transform  the  program: 

(APPEND  F00  (APPEND  NIL  BAR)) 

If  I  apply  transformation  (1)  firsL  the  resultant  program  is: 

(APPEND  F00  NIL  BAR) 

Ihcn  transformation  (2)  will  fail  to  apply.  If,  however,  1  apply  transformation  (2)  first,  the  result 
is: 


(APPEND  F00  BAR) 

Then  transformation  (1)  fails  to  apply.  In  the  first  case  (APPEND  F00  NIL  BAR)  is  the  result, 
and  in  the  other.  (APPFNO  F00  BAR).  Hence,  the  order  of  transformation  affects  not  only  which 
transformations  will  apply,  but  the  resultant  code  as  well. 


§  3.3. 1.1 


(  onlrul 


35 

A  previous  system  of  mine  dealt  with  this  problem  by  tightly  coupling  the  tr.ui»!;. filiations 
witli  the  control.  The  system  knew  exactly  when  a  transformation  was  likely  to  appl  y  hesause  the 
successful  application  of  one  transformation  would  suggest  others  that  were  likely  to  apply  next. 
lTte  transformation  system  made  only  one  pass  o\cr  the  program  tree;  it  took  the  tree  apatt  on 
the  way  down,  gathering  information  as  it  went,  then  put  it  back  together  "better "  on  the  way  up 
One  could  not  really  point  to  any  piece  of  system  code  and  say  This  is  a  ir.nwfuiinaiion";  the 
so-called  transformations  had  lost  their  identity  by  being  so  tightly  bound  up  in  die  system,  lhe 
consequences  of  this  approach  were  that  transformations  could  not  be  added,  deleted,  changed,  or 
temporarily  turned  on  or  off,  without  much  difficulty  and  groveling  around  in  the  system.  Indeed, 
there  is  some  question  as  to  whether  tin's  sort  of  system  could  correctly  be  labelled  a  transformation 
system.  It  was  rather  merely  an  “optimizing  pre-processor.” 

The  transformations  in  our  system  must  be  identifiable;  in  order  to  record  the  information 
discussed  in  section  2.4,  wc  have  to  know  when  a  transformation  is  attempted,  what  its  prerequisites 
for  application  arc,  and  whether  it  fails  or  succeeds.  Therefore,  die  current  system  instead  separates 
the  control  structure  from  die  transformations.  Furdicrmorc.  die  notion  of  interactive  optimization 
involves  giving  the  programmer  the  ability  to  turn  certain  transformations  on  or  off  and  to  choose 
the  set  or  sets  of  transformations  he  wishes  applied  in  the  first  place.  Ideally,  he  should  be  able 
even  to  write  his  own  transformations.  Clearly  he  can  do  this  only  if  this  separation  of  control  and 
transformation  is  maintained.  Hence,  we  must  severely  limit  the  knowledge  the  contro!  structure 
has  of  the  transformations  it  will  use. 

Given  a  set  of  identifiable  transformations,  independent  of  the  control  structure,  how  arc  we 
then  to  control  their  application?  Wc  must  order  not  only  die  transformations  to  be  applied,  but  the 
subtrees  to  which  they  will  apply  as  well.  For  those  two  orderings,  wc  must  dicn  choose  a  method 
of  application.  Wc  will  discuss  each  of  these  three  decisions  in  order. 


3.3. 1 . 1 .  Transformations  Within  a  Set 

Obviously  wc  want  to  choose  an  ordering  for  the  transformations  w  ithin  die  transformation  set 
which  will  produce  the  most  desirable  result  Consider  applying  the  transformation  set 

(1)  (Arithmetic  op>  Args~l>  C  <args-2>)  => 

(Arithmetic  op>  0  <args-l>  Args-2>) 

(2)  (*  0  <args> )  =>  0 

(3)  (  +  D  Args>)  =>  (+  Args>) 

(4)  (+  <arg>)  =>  <arg> 
to  die  program: 
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Input 

Transformation 

Result 

(+  7  (* 

5  3  0)) 

U) 

(+  7  (*  0  3 

(+  7  (• 

0  3  5)) 

(1) 

fails 

(+  7  (• 

0  3  5)) 

(2) 

(+  7  0) 

(+  7  0) 

(1) 

(+  0  7) 

(+  0  7) 

(1) 

fails 

(+  0  7) 

(2) 

fails 

(+  0  7) 

(3) 

(+  7) 

(+  7) 

(1) 

fails 

(+  7) 

(2) 

fails 

(+  7) 

(3) 

fails 

<  +  7) 

(4) 

7 

7 

(1) 

fails 

7 

(2) 

fails 

7 

(3) 

fails 

7 

(4) 

fails 

FINISH 

Tabi.F.3-3.  Markov  algorithm  ordering  rules  demonstrated. 

Then  neither  (2)  nor  (3)  will  apply  until  (1)  does,  but  if  (1)  applies  followed  by  (2).  (3)  will  not 
apply  unless  (1)  applies  again. 1  One  w-av  to  ensure  that  every  transformation  is  applied  as  often  as 
necessary  is  to  use  the  ordering  ailcs  of  Markov  algorithms.  If  these  rules  arc  used  in  the  above 
example.  we  obtain  the  results  given  in  fable  3-3.  The  number  of  attempts  can  be  increased 
by  changing  the  order  of  the  transformations  in  the  set.  but  for  this  example,  the  end  results 
would  be  the  same.  We  of  course  want  to  minimize  the  attempts  for  reasons  of  efficiency,  but 
we  can  derive  some  comfort  from  the  fact  that  iterating  over  the  transformations  solves  some  of 
our  problems.  Markov  algorithm  rules  arc  only  one  method  of  iteration.  We  could  specify  that  a 
transformation  is  to  be  applied  over  and  over  until  it  fails,  without  returning  to  tire  top  of  the  list 
of  transformations  between.  For  example,  the  program  (+  3  0  4  0  5  0)  would  be  changed 
to  (+000345)  after  three  successive  applications  of  (1),  then  to  (+  3  4  5)  after  three 
successive  applications  of  (3).  Or,  instead  of  requiring  the  control  structure  to  direct  die  iteration 
of  transformation  application,  we  could  place  the  burden  on  the  transformations  themselves,  That 
is.  those  transformations  which  need  to  be  iterated  would  be  responsible  for  iterating  within  them¬ 
selves. 

1  Note  that  aril limclic  oplimi/.ilion  like  this  o  very  irieky  Round  off  enor  or  e\cn  overflow  may  be  affected  by 
order  of  cvaiuaiion  and/or  atonal)'"'  I  uilherinore.  in  t  OR  I  RAN.  /  ff  0.0  may  be  used  lo  change  the  type  of  I 
Similarly,  in  Macl  ISP.  (PIUS  ( I  I XNUM- lot  Nil  T  Y  I)  0 . 0)  is  the  same  a.-.  ( F  LOAI  I) 
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Iteration  of  transformations  solves  the  problem  of  the  application  of  a  transformation  resulting 
in  code  which  can  be  changed  by  a  transformation  that  has  already  applied.  ITierc  is  still  the 
problem  in  which  the  successful  application  of  one  transformation  prevents  the  successful  applica¬ 
tion  of  another,  as  was  illustrated  in  the  APPEND  example  given  earlier.  One  solution  to  this 
problem  is  to  supply  enough  transformations  to  cover  all  the  eases  which  might  arise  as  a  result  of 
previous  transformation  applications.  That  is,  we  would  supply  a  third  transformation: 

(APPEND  <X>  NIL  <Y> )  =>  (APPEND  <X>  <Y>) 

This  solution  can  be  carried  to  extreme  though,  supplying  many  special  purpose  transformations 
with  no  general  utility  at  all.  (Why  not  simply  write  the  one  transformation  which  will  apply  to  die 
entire  program  and  produce  the  desired  result?)  Clearly,  some  care  needs  to  be  taken  when  order¬ 
ing  transformations  within  a  set;  there  is  no  system  of  application  dial  is  guaranteed  to  compensate 
for  thoughtless  ordering  on  the  part  of  the  programmer. 

Let  me  just  comment  briefly  on  one  other  ordering  strategy,  where  the  transformation  which 
applies  is  responsible  for  suggesting  the  transformation(s)  to  try  next.  That  is,  the  order  of  applica¬ 
tion  is  determined  dynamically.  There  certainly  is  much  to  be  said  for  this  mcdiod,  but  unless  it 
is  done  in  such  a  way  as  to  guarantee  that  all  diose  transformations  which  should  be  applied  are 
reachable,  it  should  only  supplement  die  methods  we  have  discussed  above.  Whether  one  specifics 
die  tiansformalion  which  is  to  be  tried  next  by  ordering  diem  w  ithin  a  set  or  by  linking  them  within 
themselves,  human  thought  must  be  used.  In  diis  system,  I  have  chosen  to  simplify  die  control 
structure  by  making  the  transformations  responsible  for  dicir  ow  n  iteration,  and  ordering  them  by 
means  of  their  placement  in  a  transformation  set. 


3. 3. 1.2.  Subtrees  of  the  Parse  Tree 

As  well  as  ordering  the  transformations  within  die  set,  we  must  choose  an  order  in  which  to 
transform  pieces  of  the  argument  program.  Several  possible  orderings  arc:  the  lexical  order  of 
the  input  text,  the  order  of  evaluation  of  the  input  program,  or  a  left-most,  depdi-first  tree  order 
[Gcschkc  1972],  The  lexical  order  of  the  input  text  is  not  a  good  choice.  In  LISP,  the  order  of 
evaluation  is  with  minor  exceptions,  a  left-most,  depth-first  tree  order.  There  arc  a  number  of 
good  reasons  to  select  the  order  of  evaluation  of  the  program  as  the  order  of  transformation  of  the 
program: 

•  Optimizing  a  program  can  be  diought  of  as  partial  evaluation  of  the  program. 

•  If  it  can  be  determined  that  a  particular  subtree  would  never  be  evaluated  then  similarly  there  is 
no  need  to  transform  it. 

•  Since  LISP  recursively  evaluates  the  arguments  of  an  expression  before  applying  the  operator 
to  those  arguments,  a  (ran formation  w  hich  applies  to  some  subtree  can  assume  that  its  subtrees 
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(DEFUN  SEXPR ! EXPAND  (SEXPR  ENVIRON  FTLIST) 

(COND  ((ATOM?  SEXPR)  SEXPR) 

((COND?  SEXPR)  ( COMO ! EXPAND  SEXPR  ENVIRON  FTL 1ST ) ) 
((DEfETTE?  SEXPR)  ( OE F£ Fit ! EXPAND  SEXPR  ENVIRON  F  TLIST) ) 
((ECALL?  SEXPR)  ( E CAE  L ! E  XPAND  SEXPR  ENVIRON  FTLIST  ) ) 

( ( ANDEXPR?  SEXPR)  ( ANDIEXPAND  SEXPR  ENVIRON  FTLIST)) 

( (OREXPR?  SEXPR)  (OR ! EXPAND  SEXPR  ENVIRON  FTLIST)) 

((QUOTE?  SEXPR)  SEXPR) 

(T  (ERROR  ' |UnparsabTe  s-expression  -  EXPAND|  SEXPR)))) 
Table  3-4.  Main  control  function  for  the  transformational  component 


have  already  been  transformed. 

To  better  illustrate  this  last  point,  consider  again  the  APPEND  example.  If  we  transform  subtrees 
in  the  order  in  which  they  will  be  evaluated,  then  the  transformations  will  apply  to  the  outer 
APPEND  only  after  they  have  applied  to  the  inner  one.  So  regardless  of  the  order  in  which  those 
transformations  occur  within  the  transformation  set.  the  transformation  which  simplifies  the  inner 
call,  (APPEND  NIL  BAR ),  will  always  apply  before  the  transformation  which  simplifies  the  outer 
call,  (APPEND  EOO  (APPENO  NIL  BAR)). 

3.3. 1.3.  Methods  of  Application 

Though  we  have  discussed  the  ordering  of  transformations  within  a  set  and  selected  an  order 
for  application  to  subtrees  of  the  parse  tree,  we  must  still  decide  on  one  of  the  following  two 
methods  of  application: 

(1)  Given  a  set  of  transformations  and  a  program  tree,  apply  each  transformation  in  its  turn  to 
all  the  subtrees  in  the  tree.  First  apply  one  transformation  everywhere,  ihen  eliminate  it, 
then  apply  the  second  transformation  everywhere,  eliminate  it,  and  continue  until  the  set  of 
transformations  is  empty. 

(2)  Given  a  set  of  transformations  and  a  program  tree,  apply  all  the  transformations  to  each  sub¬ 
tree  of  the  tree  in  its  turn.  First  apply  all  the  transformations- to  the  first  subtree  in  the  tree, 
then  move  on  to  the  next  subtree  and  apply  all  the  transformations  possible  there,  and  continue 
until  every  subtree  has  been  transformed. 

Our  transformation  system  uses  method  (2),  for  the  following  reasons: 

Since  we  have  decided  to  apply  the  transformations  to  the  subtrees  of  the  program’s  parse 
tree  in  the  order  that  those  subtrees  will  be  evaluated,  we  can  model  the  control  structure  after  the 
design  of  a  I  ISP  interpreter.  Given  some  s-expression  to  evaluate,  the  interpreter  will  first  deter¬ 
mine  the  type  of  structure  it  has  hold  of.  I.1SI'  has  a  number  of  special  forms  [Oilman  1980),  and 
all  of  the  arguments  in  these  forms  are  not  necessarily  evaluated.  For  example,  the  arguments  to 
a  conditional  expression  arc  evaluated  in  a  special  way,  as  ate  the  arguments  to  AND,  OR.  QUOTE, 
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*ATOM* 

an  atom 

•COND* 

a  conditional  expression 

•CLAUSE* 

"argument"  to  a  conditional  expression 

•DEFETTE* 

a  function  definition 

(may  be  an  internal  lambda  expression,  or  a  Macl.ISP  DE  FUN  ,  or  whatever) 

•FCALL* 

a  function  call 

(may  be  built-in  or  user  defined) 

•FUNCNAME* 

the  name  of  a  function 

•AND* 

a  conjunct 

•OR* 

a  disjunct 

•QUOTE* 

a  quoted  expression 

Table  3-5.  Structure  types. 

(DEFUM  AND! XFORM  (ANDCELL  ENVIRON  FTLIST) 

( XFORM  ' AND ! TR IM  ANDCELL  (LIST)) 

( XFORM  ' AND ! S IMPL I F  Y  ANDCELL  (LIST)) 

(XFORM  ' SEXPR ! FORM-BVAL  ANDCELL  (LIST  ENVIRON  FTLIST ) ) ) 

(DEFUN  ATOMIXFORM  (ATOMIC-EXPR  ENVIRON  FTLIST) 

(XFORM  'SEXPR! FORM-BVAL  ATOMIC-EXPR  (LIST  ENVIRON  FTLTST))) 

(DEFUN  CONDIXFORM  ( CONDCELL  ENVIRON  FTLIST) 

(XFORM  'COND! TRIM  CONOCELL  (LIST  ENVIRON  FTLIST)) 

(XFORM  ' COND ! S IMPL I F Y  CONDCELL  (LIST)) 

(XFORM  'SEXPR! FORM-BVAL  CONDCELL  (LIST  ENVIRON  FTLIST))) 

(DEFUN  DEFETTE ! XFORM  (DEFETTE  ENVIRON  FTLIST) 

(XFORM  'SEXPRIFORM-BVAL  DEFETTE  (LIST  ENVIRON  FTLIST))) 

Table  3-7.  Transformaiion  sets  for  some  structure  types. 


etc.  Just  as  the  interpreter  checks  the  type  of  expression  it  is  to  evaluate,  the  transformation  system 
looks  at  the  type  of  the  subtree  it  is  to  transform.  Based  on  that  type,  the  system  will  sec  that  the 
arguments  are  correctly  transformed  (that  is,  the  subtrees  of  the  subtree  it  is  dealing  with),  then 
will  call  those  transformations  which  can  apply  to  the  type  of  the  current  subtree.  I  have  categorized 
the  possible  s-expressions  into  different  structure  types,  and  divided  the  transformations  into  a 
separate  set  for  each  I. ISP  structure  type.  That  way.  once  the  subtrees  of  an  expression  have  been 
transformed,  the  expression  itself  is  transformed  by  the  set  of  tran formations  which  apply  to  its 
type.  The  control  mechanism  for  the  transformational  component  has  limited  knowledge  of  the 
tranfonnations  it  applies:  it  only  knows  that  for  each  structure  type  there  is  a  different  set  of 
tranformations,  and  will  apply  the  correct  set.  T  he  main  control  mechanism  is  implemented  by  the 
function  SEXPR ! EXPAND,  listed  in  Table  3-4. 
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(DEFUN  AND! EXPAND  ( ANDCELL  ENVIRON  FTLIST) 

( AND ! PRE DICAT E  ANDCELL) 

( ANOARGS! EXPAND  (AND! ARGLIST  ANDCELL)  ENVIRON  FTLIST) 

( AND ! XFORM  ANDCELL  ENVIRON  FTLIST)) 

(DEFUN  ANOARGS! EXPAND  (LOGARGS  ENVIRON  FTLIST)  ;  Slave  function  to  AND ! EXPAND 
(COND 

((ARGSINULL?  LOGARGS)  NIL) 

(T  (LEI  ((FIRST-ARG  ( ARGS ! ARG  LOGARGS))) 

( SEXPR ! EXPANO  FIRST-ARG  ENVIRON  FTLIST) 

(COND 

((TRUE?  FIRST-ARG) 

( ANOARGS ! EXPAND  ( ARGS ! RES t ARGS  LOGARGS)  ENVIRON  FTLIST)) 
((NULL?  FIRST-ARG) 

NIL)  ;Rest  logargs  not  expanded  since  won't  ever  evaluate 
(T  (ANOARGS! EXPAND  ( ARGS ! REST ARGS  LOGARGS) 

( ENV! RECORD-TEST  ENVIRON  FIRST-ARG  T  FTLIST) 
FTLIST))))))) 

(DEFUN  ATOM ! EXPAND  (ATOMIC-EXPR  ENVIRON  FTLIST) 

(ATOMtXFORM  ATOMIC-EXPR  ENVIRON  FTLIST)) 

(DEFUN  COND ! EXPAND  (CONDCELL  ENVIRON  FTLIST) 

(CLAUSES! EXPAND  ( COND ! CLAUSES  CONDCELL)  ENVIRON  FTLIST) 

(COND! XFORM  CONDCELL  ENVIRON  FTLIST)) 

(OEFUN  DEFF TTE  .'EXPAND  (DEFETTE  ENVIRON  FTLIST) 

(DEFETTEiPREDICATE  DEFETTE) 

{SEXPR1 EXPAND  ( DE FF T TE ! BODY  OEFETTF )  ENVIRON  FTLIST) 

(DEFETTE ! XFORM  DEFETTE  ENVIRON  FTLIST)) 

Table  3-6.  Control  functions  for  some  structure  types. 


Kvcry  1  ISP  structure  that  might  be  transformed  must  be  given  a  transformation  set  and  an 
opportunity  to  apply  it.  Though  it  is  possible  that  someone  might  write  a  transformation  meant  to 
apply  to  the  clause  of  a  conditional  expression,  for  example,  one  usually  thinks  of  such  a  clause  as 
part  of  the  larger  *COND*  structure,  rather  than  a  structure  type  itself.  Rather  than  put  an  limita¬ 
tions  on  transformations,  however.  I  chose  to  give  all  such  “substructures"  structure  types  of  their 
own.  T  he  structure  types  I  have  defined  can  be  seen  in  T  able  3-5.  Not  all  of  them  arc  recognized 
by  SEXPR !  EXPAND,  since  some  of  them  are  not  evaluable  independently  of  their  superstructure. 
Hach  structure  type  has  its  own  control  function.  Some  of  these  control  functions  simply  call  the 
appropriate  control  function  for  its  structure's  subtree,  followed  by  a  call  to  the  transformation 
set  (sec  Table  3-6  and  Table  3-7),  but  a  few  types  need  slightly  more  processing.  The  *FCALL* 
stiucturc  type,  for  example,  consists  of  a  function  and  an  argument  list.  The  function  may  be  either 
be  the  name  of  a  function  (and  thus  a  special  atom  of  type  *1  UNCNAME»).  in  which  case  it  is  a 
candidate  for  procedure  integration,  or  it  may  be  ail  internal  lambda  definition.  Controlling  the 
transformation  of  internal  lambda  definitions  can  be  a  bit  tricky .  especially  when  it  comes  to  keep¬ 
ing  the  information  which  has  been  gathered  straight  concerning  the  changes  in  variable  names 
brought  about  by  lambda  binding.  Hut  that  is  a  subject  for  the  next  section. 
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3.3.2.  Information  Gathering 

Information  gathering  refers  to  the  ascertaining  of  those  truths  about  a  piece  of  code  which 
form  its  context.  Local  transformations  such  as  the  rules  (CAR  (CONS  <X>  <Y>))  =>  <X> 
and  ( EQUAL  <X>  <X> )  =>  T  don't  need  to  know  anything  about  the  context  of  die  code  they 
are  transforming  [Bagwell  1970].  But  global  transformations  do;  their  validity  depends  on  the 
nature  of  the  surrounding  code  [Schaefer  1973],  Since  we  have  decided  to  transform  expressions  in 
the  order  in  which  they  are  evaluated,  the  transformation  system  will  have  walked  down  the  parse 
tree  as  far  as  the  expression  it  is  currently  transforming.  That  means  it  can  be  aware  of  the  context 
of  an  expression  by  the  time  it  gets  to  the  expression,  and  merely  needs  a  method  for  recording  that 
context. 


3.3.2.I.  Clarifying  the  Need 

The  substitution  rule  (OR  <X>  T)  =>  T  is  not  always  correct.  In  Macl. ISP,  at  least,  OR 
returns  the  value  of  the  first  argument  which  is  true.  Thus,  (OR  3  T )  returns  3,  not  T.  However, 
if  this  test  occurs  in  a  predicate  position,  such  as  the  predicate  of  a  conditional  clause,  only  the 
boolean  value  of  the  expression  is  needed  and  the  transformation  of  { OR  3  T )  to  T  would  be 
correct.  Knowing  whether  or  not  an  expression  is  in  predicate  position  is  a  type  of  what  I  call 
“special”  contextual  information,  as  opposed  to  “common”  contextual  information.  Special  contex¬ 
tual  information  is  true  only  of  a  particular  instance  of  an  expression.  Just  because  (OR  <X>  T) 
occurs  in  predicate  position  at  one  point  in  code,  doesn’t  imply  that  other  instances  of  that  expres¬ 
sion  do  also.  Common  contextual  information  refers  to  truths  pertaining  to  every  occurrence  of  a 
particular  expression.  Consider  optimizing  the  code: 

(DEFUN  EXAMPLE  (FOO  BAR  BAZ) 

(CONO  ((ATOM  FOO)  NIL) 

( ( ISINOEXED  FOO)  (PROCESS-INDEX  FOO  BAR  BAZ)) 

(  ...  (GET-PART  FOO)  ...  ) 

(T  ...  ))) 

(DEFUN  ISINOEXED  (X) 

(AND  (NOT  (ATOM  X)) 

(ATOM  (CAR  X)) 

(NUMBERP  (CAR  X)))) 

(DEFUN  GET-PART  (Y) 

(COND  ((ATOM  Y)  NIL) 

((AND  (ATOM  (CAR  Y)) 

(NUMBERP  (CAR  Y) ) ) 
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(CADR  Y)) 

(T  (CAR  Y)))) 

When  ISINDEXED  is  expanded  in-line,  the  definition  of  EXAMPLE  will  appear  as: 

(DEFUN  EXAMPLE -PROGRAM  ( F00  BAR  BAZ) 

(COND  ((ATOM  FOO)  NIL) 

((AND  (NOT  (ATOM  FOO)) 

(ATOM  (CAR  FOO)) 

(NUMBERP  (CAR  FOO))) 

(PROCESS-INDEX-PROGRAM  FOO  BAR  BAZ)) 

(  ...  (GET-PART  FOO)  ...  ) 

(T  ...  ))) 

The  test  (ATOM  FOO)  unnecessarily  appears  twice.  The  second  clause  of  the  conditional  ex¬ 
pression  can  only  be  evaluated  if  the  predicate  of  the  first  clause  returned  NIL.  Thus,  at  the 
point  where  ISINOEXEO  is  expanded,  it  is  known  that  (ATOM  FOO)  is  NIL.  Similarly,  at  the 
time  that  PROCESS- INOEX-PROGRAM  is  evaluated,  it  will  be  known  that  FOO  is  not  an  atom, 
(CAR  FOO)  is  an  atom,  and  (CAR  FOO)  is  a  number.  And,  when  (GET-PART  FOO)  is  ex¬ 
panded,  it  is  known  that  FOO  is  not  an  atom,  and  that  (CAR  FOO)  is  not  both  an  atom  and  a 
number.  If  at  any  time  a  test  is  performed  whose  outcome  is  already  known,  we  would  like  to  have 
the  ability  to  eliminate  that  duplicate  test.  Thus,  after  transformation,  EXAMPLE  should  be: 

(DEFUN  EXAMPLE-PROGRAM  (FOO  BAR  BAZ) 

(COND  ((ATOM  FOO)  NIL) 

((AND  (ATOM  (CAR  FOO)) 

(NUMBERP  (CAR  FOO))) 

(PROCESS-INDEX-PROGRAM  FOO  BAR  BAZ)) 

(  ...  (CAR  FOO)  ...  ) 

(T  ...  ))) 

I  should  mention  here  quickly  one  caution  about  global  transformations.  Most  high-level 
languages  are  impotent  without  the  use  of  side-effects.  But  side-effects  wreak  havoc  for  global 
transformations,  which  must  then  constantly  be  on  tire  lookout  for  changes  in  the  values  of  current 
variables.  Just  because  FOO  is  not  an  atom  at  one  point  in  the  code  doesn't  guarantee  that  it  is 
later,  since  (SETQ  FOO  <whatever>)  (or  something  similar)  could  occur  in  the  meantime.  I 
have  currently  made  no  provisions  for  the  transformation  of  code  which  may  contain  side-effects. 
The  transformational  component  of  this  system  assumes  that  the  I.1SP  code  it  operates  on  is  totally 
applicative.  I  can  defend  this  seeming  lack  of  power  on  two  counts: 

•  LISP  is  one  high-level  language  that  manages  to  accomplish  a  great  deal  even  when  limited  to 
non-sidc-cffccting  functions.  So  it  is  not  completely  unreasonable  to  limit  the  transformation 
system  in  this  way. 
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(DEFUN  AND! PREDICATE  (ANDCELL) 

( SEXPR IMAPCAR  (FUNCTION  PREOI F  Y )  ( AND! ARGL 1ST  ANDCELL)) 
(COND  ((NOT  (PRED?  ANDCELL)) 

(OEPREDIE Y  ( ARGS ! LAST-ARG  ( AND! ARGLIST  ANDCELL)))))) 


(OEFUN  DE FE TIE ! PREDICATE  (DEFETTE) 

(COND  ((PRED?  DEFETTE)  (PREDIFY  (DEFETTE !BODY  DEFETTE))))) 


Table;  3-8.  Propagation  of  predicate  position  information. 


•  The  accountable  component  of  our  system  is  in  no  way  dependent  on  this  constraint.  It  merely 
records  the  transformations  made  by  the  transformational  component,  and  couldn’t  care  less 
about  the  restrictions  on  or  justification  for  those  transformations.  Hie  emphasis  of  this  thesis 
is  on  the  accountable  component;  the  transformational  component  1  am  describing  has  been 
implemented  merely  to  demonstrate  the  use  of  the  other. 


3.3.2.2.  The  Environment 

In  order  to  allow  global  transformations  in  our  system,  we  record  both  special  and  common 
contextual  information.  Special  contextual  information  docs  not  present  much  of  a  problem; 
we  simply  augment  the  internal  form  of  the  program  by  storing  properties  of  expressions  along 
with  the  expression  itself.  When  a  conditional  expression  is  encountered,  the  predicate  position 
property  may  be  added  to  all  the  predicate  expressions  within  the  conditional.  If  an  AND  expres¬ 
sion  is  in  predicate  position,  than  all  its  arguments  arc  also.  If  not.  all  but  its  last  argument 
is  in  predicate  position,  and  so  on.  lire  code  in  Table  3-6  contains  function  calls  of  the  form 
<type>  !  PREDICATE;  their  definitions  arc  given  in  Table  3-8.  Common  contextual  information, 
on  the  other  hand,  needs  to  be  recorded  in  such  a  way  that  code  lower  in  the  tree  can  access  it.  Wc 
will  do  this  by  maintaining  an  environment,  much  like  a  MSP  interpreter  docs.  Ihen  whenever  a 
transformation  requests  information  about  some  piece  of  code,  the  environment  is  queried. 

Tach  member  of  the  environment  consists  of  some  MSP  expression  w'hosc  truth  value  is 
known,  its  truth  value,  and  a  justification  for  that  value.  The  environment  acquires  more  knowledge 
every  umc  it  encounters: 

•  a  conditional  expression, 

•  an  AND  expression, 

•  an  OR  expression,  or 

•  explicit  assertions  in  the  code. 
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A  conditional  of  the  form: 


(cond  (<pred-l> 
( <pred-2> 
(<pred-3> 


<exi  ,form-l>) 
<exitform-2>) 
<exitform-3>) 


) 


can  assume  at  each  location  the  truths  given: 

(cond  (? 

((<pred-l>  false] 

( [<pred-l>  false,  <prcd-2>  false] 


|<pred-l>true]  ) 

[<pred-2>  true,  <prcd-l>  false]  ) 

[<pred-3>  true,  <pred-2>  false,  <prcd-l>  false]) 


) 


Table  3-9.  Propagation  of  information  within  a  conditional  expression. 


As  we  saw  in  the  above  section,  tests  performed  within  a  conditional  expression  result  in 
information  which  can  then  be  distributed  within  that  expression.  Tabic  3-9  illustrates  the  infor¬ 
mation  which  can  be  distributed.  CLAUSES! BRANCH  is  the  function  which  knows  how  to  trans¬ 
form  conditional  expressions.  Conditionals  are  a  special  form  in  LISP  because  their  “arguments” 
arc  not  necessarily  all  evaluated.  Thus,  instead  of  transforming  all  the  clauses  right  away, 
CLAUSES !  BRANCH  transforms  only  the  first  predicate  of  the  first  clause  in  the  list  of  clauses.  If 
that  predicate  is  determined  to  be  true,  CLAUSES !  BRANCH  transforms  the  rest  of  the  clause  and 
quits,  just  like  the  LISP  interpreter  w  ould  do.  If  the  predicate  is  false,  it  ignores  the  rest  of  the 
clause  and  calls  itself  on  the  remainder  of  the  clauses.  If  the  predicate  can  not  be  determined  true 
or  false,  then  it  must  transform  both  the  rest  of  the  clause  and  die  remaining  clauses,  but  it  does 
this  with  an  updated  environment.  Table  3-10  contains  a  listing  of  CLAUSES! BRANCH.  Notice 
die  two  calls  to  ENV!  RECORD-TEST  in  the  definition  of  CLAUSES  !  BRANCH  which  occur  when 
die  current  predicate  cannot  be  determined  to  be  true  or  false.  In  that  case,  the  environment  used 
for  the  transformation  of  the  rest  of  the  clause  should  contain  the  information  that  the  current 
predicate  is  true,  and  the  environment  used  by  die  rest  of  the  clauses  in  the  conditional  expres¬ 
sion  should  know  diat  the  current  predicate  is  false.  ENV!  RECORD-  TEST  lakes  an  environment, 
the  predicate  expression,  and  the  predicate's  outcome,  and  returns  a  new  environment  with  the 
additional  information  recorded. 
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(DtfUN  CLAUSES!F XPAND  (CONDLISI  ‘CLAUSES-ENVIRON.  ‘CLAUSE S-ETL  1ST*) 

(DECLARE  (SPECIAL  ‘CLAUSES-ENVIRON*  ‘CLAUSES- F TL 1ST* ) ) 

(COND 

( ( CLAUSES ! NULL?  CONDLIST)  NIL) 

(T  (LET  ( ( X T SI  (CLAUSE !  PRED  ( CL AUSES ! CL AUSE  CONDLIST)))) 

(PREDIFY  XTST ) 

( SE XPR ! EXPAND  XTST  »CLAUSES-ENVIRON‘  ‘CLAUSES- F TL 1ST* ) 

(COND 

((TRUE?  XTST) 

Rest  of  clauses  not  expanded  since  evaluation  stops  here. 

( SE  XPR !MAPCAR  (FUNCTION 

(LAMBDA  (SEXPR) 

(SEXPRiEXPAND  SEXPR 

•CLAUSES-ENVIRON* 

•CLAUSES-FTLIST*))) 

(CLAUSE ! EXITFORM  ( CL AUSES ! CL AUSE  CONDLIST)))) 

((NULL?  XTST) 

;;  Rest  of  clause  not  expanded  since  it  will  never  be  returned 
(CLAUSES ! EXPAND  ( CLAUSES ! CLAUSES  CONDLIST) 

•CLAUSES-ENVIRON* 

•CLAUSES-FTLIST*)) 

(T  (LET 

((•LET-TLIST* 

( ENV (RECORD -TEST  ‘CLAUSES-ENVIRON*  XTST  T  ‘CLAUSES-FTLIST*))) 
(DECLARE  (SPECIAL  ‘LET-TL IST‘ ) ) 

( SEXPR ! MAPCAR  (FUNCTION 

(LAMBDA  (SEXPR) 

(SEXPR ! EXPAND  SEXPR 

‘LET-TIIST. 

•CLAUSES-FTLIST.))) 

(CLAUSEJEXITFORM  ( CL AUSES ! CLAUSE  CONDLIST)))) 
(CLAUSES! EXPAND  (CLAUSES ! CLAUSES  CONDLIST) 

(ENV! RECORD-TEST  ‘CLAUSES-ENVIRON* 

XTST 

NIL 

•CLAUSES-FTLIST.) 

•CLAUSES-FTLIST.))))))) 

Taplp.  3-10.  Controlling  function  for  conditional  clauses. 


After  transforming  any  argument  of  an  AND  expression,  transformation  of  die  remaining  ar¬ 
guments  takes  place  with  the  assumption  that  the  preceding  argument  is  true.  Similarly  for  an  OR 
expression,  the  transformation  of  any  argument  may  assume  that  all  preceding  arguments  are  false. 
The  environment  used  for  the  transformation  of  these  arguments  is  updated  and  maintained  using 
the  same  functions  described  above. 

Assertions  consist  of  an  s-expression  and  a  list  of  truths  to  be  assumed  by  that  expression.  An 
assertion  may  be  hand  coded  by  the  programmer  or  generated  automatically  by  a  transformation. 
When  an  assertion  is  encountered  by  the  main  control  function.  SEXPR !  EXPAND,  it  records  the 
truths  on  the  current  env iroment  and  proceeds  to  transform  the  enveloped  text.  Thus  an  assertion 
constitutes  a  new'  structure  type;  we  must  provide  for  it  by  redefining  SEXPR !  EXPAND,  as  seen  in 
Table  3-11. 
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(DEFUN  SEXPR ! EXPAND  (SEXPR  ENVIRON  FIL1ST) 

(COND  ((ATOM?  SEXPR)  SEXPR) 

((COND?  SEXPR)  ( COND! EXPAND  SEXPR  ENVIRON  FTL  1ST ) ) 

((DEFETTE?  SEXPR)  ( DE FETTE ! E XPAND  SEXPR  ENVIRON  FTLIST)) 

( ( F CALL?  SEXPR)  ( ECALL ! EXPAND  SEXPR  ENVIRON  FTLIST)) 

( ( ANDEXPR?  SI XPH )  ( AND ! EXPAND  SEXPR  ENVIRON  FTLIST)) 

( ( OREXPR?  SEXPR)  (OR1EXPANO  SEXPR  ENVIRON  FTLIST)) 

((QUOTE?  SEXPR)  SEXPR) 

«  ((ASSERT?  SEXPR)  ( ASSERT ) EXPAND  SEXPR  ENVIRON  FTLIST)) 

(T  (ERROR  ' |Unparsable  s-enpression  -  EXPAND|  SEXPR)))) 

(OEFUN  ASSERT ! EXPANO  (ASSERTION  ENVIRON  FTLIST) 

(SEXPR ! EXPAND  ( ASSERT ! SEXPR  ASSERTION) 

(ENV!RECORD-TEST 

ENVIRON 

( SEXPR ! EXPAND  ( ASSERT ! TRUTHS  ASSERTION)  ENVIRON  FTLIST) 
T 

FTLIST) 

FTLIST)) 

Tabu- 3-11.  Providing  for  assertions. 


The  function  ENV!  RECORD-TEST  has  the  responsibility  of  recording  a  test  on  the  environ¬ 
ment.  Rather  than  blindly  recording  the  test  given  it,  the  function  attempts  to  break  the  test  down 
into  as  many  of  its  parts  as  possible.  For  example,  if  ENV!  RECORD-TEST  is  told  to  record  as  true 
the  test  (AND  P  Q  R),  it  will  instead  construct  and  record  the  individual  entries:  P  is  true, 
Q  is  true,  and  R  is  true.  If  it  is  told  to  record  the  fact  that  (AND  A  B  C)  is  false,  it 
first  looks  on  the  current  environment  to  see  if  any  of  A,  B,  or  C  arc  known  to  be  true.  If  so, 
ENV! RECORD-TEST  can  simplify  the  given  test  and  record  that.  Suppose  it  secs  that  B  is  true. 
Then  either  A  or  C  must  have  made  ( AND  ABC)  false.  Thus  it  records  as  false  the  expression 
(AND  A  C).  A  predicate  of  the  form  (NULL  <x>)  or  (NOT  <x>)  is  stripped  of  its  negation 
and  the  outcome  toggled  before  the  expression  is  recorded  on  the  environment.  Hence,  if  it  is 
known  that  (NOT  FOO)  is  true,  the  entry  FOO  is  false  is  made  on  the  environment 

These  rules  for  recording  information  enforce  a  canonical  fonn  for  entries  in  the  environment 
that  most  efficiently  provides  information.  Suppose  we  wished  to  determine  whether  or  not 
(AND  0  R)  was  true.  We  can  search  the  environment  and  learn  that  0  is  taic  and  that  R  is 
true,  thus  (AND  0  R)  must  be  true.  If,  when  (AND  P  0  R)  was  recorded  as  true  above,  we 
had  simply  entered  the  entire  predicate,  we  would  have  had  a  harder  time  \ crifying  (AND  Q  R). 
Instead  of  tuning  to  perform  the  logic  every  time  something  is  looked  up.  we  record  the  smallest 
pieces  possible  in  the  first  place.  A  listing  of  ENV!  RECORD-TST  is  given  in  Table  3-12. 

I  should  mention  one  thing  more  about  recording  information  on  the  environment:  occasion¬ 
ally  there  arc  predicates  recorded  whose  truth  or  falsity  may  imply  other  truths.  For  example,  if  a 
certain  expression  is  known  to  be  false  (that  is.  null),  then  it  is  necessarily  an  atom,  and  conversely 
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(DEFUN  ENV! RECORD- TEST  (ENVIRON  TST  OUTCOME  FTLIST) 

(COND 

((NULL  TST)  ENVIRON) 

((ATOM?  TST) 

(ENVIADD-TEST  ENVIRON 

( ENV-EL ! CREATE  ( FALIST ! SUBLIS  FTLIST  TST)  OUTCOME  NIL))) 

( (ANDEXPR?  TST) 

(COND  (OUTCOME  ;  recording  a  true  AND  test 

(ENVIRECORD-TESTS  ENVIRON  ( AND ! ARGL 1ST  TST)  OUTCOME  FTLIST)) 

(T  (ENVIADD-TEST  ;  recording  a  false  AND  test 
ENVIRON 
( ENV-EL ICREATE 
(FALIST ISU8L IS 
FTLIST 
(LET 

((REFINED 

(ARGSIREFINE  (AND! ARGLIST  TST)  OUTCOME  ENVIRON  FTLIST))) 
(COND  ( ( ARCS !ONE?  REFINED)  (ARGSIARG  REFINED)) 

(T  (ANDiCREATE  REFINED))))) 

OUTCOME 

NIL))))) 

( (ORE XPR?  TST) 

(COND  (OUTCOME  ;  recording  a  true  OR  test 

(ENVIAOD-TEST 
ENVIRON 

(ENV-EL ! CREATE 
(FALIST ! SUBLIST 
FTLIST 
(LET 

((REFINEO 

(ARGSIREFINE  (OR! ARGL I  ST  TST)  OUTCOME  ENVIRON  FTLIST))) 
(COND  ( ( ARGS ! ONE?  REFINED)  (ARGSIARG  REFINED)) 

(T  (ORICREATE  REFINED))))) 

OUTCOME 

NIL))) 

(T  ;  recording  a  false  OR  test 

(ENVIRECORD-TESTS  ENVIRON  ( OR  I ARGLIST  TST)  OUTCOME  FTLIST)))) 

( (NOTEXPR?  TST)  ;  strip  off  NOT  and  toggle  outcome 

(ENVIRECORD-TEST  ENVIRON 

(ARGSIARG  ( FCALL I ARGLIST  TST)) 

(NOT  OUTCOME) 

FTLIST)) 

(T  (ENVIAOD-TEST  ENVIRON 

(ENV-ELICREATE  ( FAL 1ST  I SUBL IS  FTLIST  TST)  OUTCOME  NIL))))) 

Table  3-12.  Recording  information  on  the  environment 


so.  Thus,  whenever  such  a  predicate  is  recorded,  any  implications  are  recorded  as  well.  This  is  done 
by  the  function  ENVIADD-TEST. 
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3.3. 2. 3.  The  Index  of  Bindings 

The  current  index  of  bindings  is  maintained  in  the  variable  FTLIST.  This  index  associates 
with  each  current  internal  lambda  variable  its  value  in  terms  of  top  level  variables.  More 
informally,  it  is  an  association  list  of  dummy  arguments  and  actual  arguments.  It  provides  for  die 
v  irtual  substitution  of  actual  arguments  for  formal  arguments  w  ithin  any  number  of  nested  lambda 
expressions,  so  that  information  recorded  at  one  level  is  accessible  by  transformations  performed  at 
deeper  levels.  l  et's  look  at  an  example. 

(DEFUN  MYSEARCH  (ENV  TST) 

(COND  ((EMPTY?  ENV)  NIL) 

((EQUAL  TST  (GET-TST  (GET-ENTRY  ENV)))  (GET-ENTRY  ENV)) 

(T  (MYSEARCH  (GET-REST  ENV)  TST)))) 

(DEFUN  EMPTY?  (ENVIRON) 

(NULL  ENVIRON)) 

(DEFUN  GET-ENTRY  (ENVIRONMENT) 

(COND  ((NULL  ENVIRONMENT) 

(ERROR  '|Can't  ask  for  element  of  empty  environment))) 

(T  (CAR  ENVIRONMENT)))) 

Suppose  that  a  set  of  optimizing  transformations  is  applied  to  die  code  above.  When  MYSEARCH  is 
optimized,  procedure  integration  of  EMPTY?  will  occur  in  the  first  clause  of  die  conditional.  While 
transforming  the  remaining  clauses,  then,  it  is  known  that  (NULL  ENV)  is  false.  (We  always 
record  the  transformed  version  of  the  predicate,  not  the  untransfonned  version,  as  the  former  may 
be  simpler;  if  it  is,  no  information  will  be  lost  by  recording  die  simpler  version  since  diat  version 
will  have  been  derived  from  information  already  on  the  environment.)  During  transformation 
of  the  second  clause,  procedure  integration  of  GET-ENTRY  will  take  place.  The  definition  of 
MYSEARCH.  will  dicn  look  like; 

(DEFUN  MYSEARCH  (ENV  TST) 

(COND 

((NULL  ENV)  NIL) 

((EQUAL 

TST 

(GET-TST 

((LAMBDA  (ENVIRONMENT) 

(COND 

((NULL  ENVIRONMENT) 

( FRROR 

'|Can't  ask  for  element  of  empty  environment))) 
(T  (CAR  ENVIRONMENT)))) 

ENV))) 

(GET-ENTRY  ENV)) 
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(DEFUN  FCALUEXPAND  ( f CALL  ENVIRON  FRIST) 

( ECALL ! PREDICATE  f CALL ) 

(ARGLIST JEXPAND  ( FCAL L ! ARGL 1ST  ECALL)  ENVIRON  FTLIST  )  ;  Expand  the  arguments 
(COND  ( ( F  UNCNAME  ?  ( f  CALL ! FUNC  FCALL ) ) 

( FUNCNAMl ! XFORM  ( EC ALL ! F UNC  FCALL)  ENVIRON  FTLIST))) 

;;  note  that  because  the  above  transformer  side-effects,  a  given  fcaFl 
;;  may  pass  both  the  previous  and  following  tests,  (i.e.  f UNCNAME ! SUB  -OFF 
;;  may  make  a  defcell  out  of  the  funchead  of  the  fcall.) 

(COND  ((DEFETTE?  ( F CALL ! FUNC  FCALL)) 

(DEFETTE ! EXPAND  ( FCALL ! FUNC  FCALL) 

ENVIRON 

(FALISTIADD-PAIRS  ;  the  new  ftlist 
(  F AL 1  ST ! CRE  ATE 

(BINDING! DARGL 1ST 

(DEFETTE ! B 1 ND I NG  ( F CALL ! F UNC  FCALL))) 

( F AL 1ST ! SUBL 1 S  FTLIST  ( F CALL ! ARGL 1ST  FCALL))) 
FTLIST)) 

(DEFCALL ! XFORM  FCALL  ENVIRON  FTLIST))) 

(COND  ((NOT  (FCALL?  FCALL))  FCALL)  ;  FCALL  may  no  longer  be  type  .FCALL« 

(T  (FCALL ! XFORM  FCALL  ENVIRON  FTLIST)))) 

Tabu-:  3-13.  Control  function  for  the  Function  Call  structure  type. 


(T  (MYSEARCH  (GET-REST  ENV)  TST ) ) ) ) 

While  tranformation  of  the  body  of  the  internal  lambda  expression  is  taking  place,  information 
about  die  variable  ENVIRONMENT  may  be  gathered  or  requested.  But  only  expressions  within  the 
lambda  function  can  contribute  to  or  benefit  from  that  information,  since  ENVIRONMENT  is  a  local 
variable.  We  need  some  means  of  communicating  with  variables  outside  the  lambda  function.  That 
is,  sonic  means  of  translating  between  ENVIRONMENT  and  ENV.  I'hc  index  of  bindings  provides 
that  for  us.  Before  anything  is  recorded  or  searched  for  in  die  environment,  it  is  translated  into 
the  equivalent  expression  using  lop  level  \ariablc  names,  by  means  of  the  index  of  bindings.  Thus 
index  is  updated  whenever  tin  internal  lambda  function  is  entered.  Thus,  for  example,  die  index 
is  empty  during  transformation  of  the  outer  conditional  above,  but  associates  the  dummy  variable 
ENVIRONMENT  with  the  actual  argument  ENV  during  transformation  of  the  inner  conditional 
(which  occurs  inside  die  lambda  function). 

When  an  internal  lambda  expression  is  encountered,  the  index  of  bindings  is  updated  as 
follows: 

(1)  Using  the  current  index  of  bindings,  translate  the  actual  arguments  of  the  new  lambda  expres¬ 
sion  into  strictly  top  level  variable  references  (since  they  may  contain  references  to  earlier 
interna!  lambda  expression  variables). 

(2)  Create  an  association  list  of  die  dummy  arguments  used  in  die  new  lambda  expression,  and  the 
translated  actual  arguments  from  (l). 
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(3)  Append  the  association  list  created  in  (2)  onto  the  front  of  the  current  index  of  bindings,  to 
create  the  new  index  of  bindings.  (‘I lie  association  list  from  (2)  must  go  on  the  front  so  that 
if  any  lambda  variable  names  arc  reused,  the  new  definitions  will  shadow  die  old  ones  on  the 
index  of  bindings.) 

The  code  which  performs  these  steps  is  part  of  the  definition  of  FCALL  .'EXPAND,  which  can  be 
seen  in  Table  3-13. 

Why  are  we  limited  to  the  virtual  substitution  made  possible  by  die  index  of  bindings,  as 
opposed  to  just  always  substituting  actual  arguments  for  formals  and  therefore  avoiding  the  com¬ 
munication  problem?  Because  only  transformations  have  the  prerogative  of  performing  substitu¬ 
tions  in  the  code.  Ihe  environment  and  its  index  is  a  compendium  of  information  provided  as  a 
service  to  those  transformations  which  require  global  knowledge  of  the  expression  they  are  operat¬ 
ing  on.  A  transformation  may  or  may  not  perform  a  substitution,  but  that's  none  of  the  business  of 
the  control  mechanism. 


3.4.  The  Transformations 

To  complete  our  discussion  of  die  transformational  component.  I  need  to  discuss  several 
aspect*  of  transformations.  In  order  for  someone  to  write  their  own  transformation  for  the  system, 
they  must  know 

(a)  die  sy  ntactic  form  in  which  the  system  expects  to  get  the  transfonnation, 

(b)  in  which  transformation  set  to  put  the  transfonnation,  and 

(c)  die  expected  inputs  and  outputs  of  the  transformation. 

3.4.1.  Form 

Wh.u  constitutes  a  transformation?  In  previous  sections  I  have  used  a  substitution  rule  form 
for  transformations,  such  as  (NULL  NIL)  =>  T.  While  it  is  possible  to  devise  a  system  in 
which  transformations  are  actually  given  in  that  form,  it  involves  some  complex  pattern  matching 
capabilities.  Consider  die  software  necessary  to  find  an  application  of  the  transformation: 

(COND  <clauses-l>  (NIL  <exitform>)  <clauses-2>) 

=>  (CONO  <clauses-l>  <clauses-2>) 

where  <clauses-l>  and  <clauses-2>  can  match  any  number  of  (or  zero)  clauses.  While 
such  pattern  matching  is  possible  and  certainly  has  been  done  before  [Boyle  1970],  it  requires 
sophisticated  control  mechanisms  (and  we  re  trying  to  keep  our  control  simple,  remember?). 

I  urthermore.  not  all  transtorinations  arc  cosily  expressed  as  substitutions.  In  particular,  global 
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(DEFUN  F CALL ! S IMPL I F  Y  ( FCALL ) 

(COND  ((AND  (CAREXPR?  FCALL) 

(CONSEXPR?  ( FCALL ! F IRST -ARG  T CALL ) ) ) 

(RESPOND  T  ( FCALL ! FIRST -ARC  ( f CAL L ! F IRST -ARG  'CALL)))) 

(T  (RESPOND  NIL  FCALL)))) 

Tablk 3-14.  Implementation  of  (CAR  (CONS  <X>  <Y>))  =><X>. 


transformations  cannot  be  expressed  in  that  way  since  their  applicability  conditions  may  be  more 
complex  than  a  simple  pattern  match. 

One  might  consider  beefing  up  a  substitution  rule  form  to  include  other  information  necessary 
to  describe  the  transformation.  Pre-  and  post-  assertions  could  be  tacked  on.  as  well  as  predicates 
to  determine  whether  or  not  this  transformation  will  really  buy  anything  (an  evaluation  of  its 
"goodness”  in  this  situation,  called  "win”  predicates  in  (I.ovcman  1977]),  and  perhaps  even  a  list  of 
transformations  to  try  next.  There  are  all  sorts  of  possibilities.  However,  for  my  current  purposes 
1  want  to  concentrate  on  accountability  and  keep  the  control  for  the  transformational  component 
very  simple.  Therefore,  I  have  opted  for  transformations  that  know  how  to  apply  themselves 
(otherwise  known  as  procedures). 

In  our  system,  once  a  transformation  has  been  called  it  is  responsible  for  figuring  out  whether 
it  applies  or  fails,  and  for  returning  a  result  to  the  control  function  which  called  it.  In  this  thesis, 
I  refer  to  the  code  which  implements  a  transformation  as  a  “transformer".  Table  3-14  shows 
the  transformer  which  implements  the  transformation  (CAR  (CONS  <X>  <Y>))  =>  <X>.  The 
control  function  passes  to  the  transformer  the  expression  to  be  transformed,  and  the  uansformer 
responds  with  either  T  or  NIL,  indicating  whether  or  not  it  succeeded,  followed  by  the  resultant 
expression. 

One  of  the  benefits  of  implementing  transformations  as  procedures  is  that  a  single  transformer 
may  implement  more  than  one  transformation  rule.  If  similar  rules  are  grouped  together  in  one 
transformer  they  can  share  some  of  the  work  needed  to  determine  if  they  apply.  The  transformer 
FCALL ! SIMPLIFY  given  in  Table  3-14  is  expanded  in  Table  3-15  to  include  the  following  trans¬ 
formation  rules: 

(CAR  (CONS  <A>  <B>) )  O  <A> 

(CAR  (LIST  <A>  ...))  *>  <A> 

(CDR  (CONS  <A>  <B>) )  »>  <B> 

(CDR  (LIST  <A>  <B>  ...))  =>  (LIST  <B>  ...) 

(APPEND  NIL  <X>)  =>  <X> 

(APPEND  <X>  NIL)  *>  <X> 
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(DEFUN  FCALLISIMPLIFY  (FCALL) 

(COND 

;;  (CAR  (CONS  <A>  <B>)'  ->  <A> 

; ;  (CAR  (LIST  <A>  .  .  .  ))  ->  <A> 

((ANO  (CAREXPR?  FCALL) 

(NOT  (ATOM?  (FCALLIFIRST-ARG  FCALL)))) 

(CONO  ((CONSEXPR?  ( FCALL ( F IRST-ARG  FCALL)) 

(RESPOND  T 

(FCALUFIRST-ARG  {  FCALL  !  F IRST -ARG  FCALL)) 

(JUSTICREATE  '|(CAR  (CONS  <A>  <B>))  ->  <A>|  NIL))) 

( ( LISTEXPR?  (FCALL !  F IRST  -ARG  FCALL)) 

(RESPOND  T 

( FCALL ! F IRST-ARG  (FCALL ! FIRST-ARG  FCALL)) 

(JUST ! CREATE  ' | ( CAR  (LIST  <A>  ...))  *>  <A>|  NIL))) 

(T  (RESPONO  NIL 

FCALL 

(JUSTICREATE  '|1t  Isn't  CAR  of  CONS  or  LIST. |  NIL))))) 

;;  (COR  (CONS  <A>  <B>))  ->  <B> 

;;  (COR  (LIST  <A>  <B>  ...))  ->  (LIST  <B>  ...) 

((AND  (CDREXPR?  FCALL) 

(NOT  (ATOM?  (FCALL! FIRST-ARG  FCALL)))) 

(COND  ((CONSEXPR?  (FCALUFIRST-ARG  FCALL)) 

(RESPONO  T 

( FCALL !SECONO-ARG  ( FCALL ! F IRST-ARG  FCALL)) 

(JUSTICREATE  '|(CDfl  (CONS  <A>  <B>))  ->  <B>|  NIL))) 

((LISTEXPR?  ( FCALL ! F IRST -ARG  FCALL)) 

(XFORM-SLAVE  ' SEXPR ! NOTHING 

(FCALLIFIRST-ARG  (FCALL! FIRST-ARG  FCALL)) 

(LIST  (JUSTICREATE  '|it  was  COR'd  over.|  NIL)) 

FCALL) 

(RESPOND 

T 

( FCALL ! FIRST-ARG  FCALL) 

(JUSTICREATE  '|(CDR  (LIST  <A>  <B>  ...))  ->  (LIST  <B>  ...)|  NIL))) 

(T  (RESPOND  NIL 

FCALL 

(JUSTICREATE  ' | Ft  isn't  CDR  of  CONS  or  LIST . |  NIL))))) 

; ;  (APPEND  NIL  <X>)  ->  <X> 

((AND  ( APPENDEXPR?  FCALL) 

(NULL?  ( FCALL  I  FIRST-ARG  FCALL))) 

(RESPONO  T 

( FCALL  I SECOND-ARG  FCALL) 

(JUSTICREATE  '|(APPEND  NIL  <X>)  ->  <X>  |  NIL))) 

; ;  (APPEND  <X>  NIL)  ->  <X> 

((AND  ( APPENOE XPR?  FCALL) 

(NULL?  (FCALL I SECOND-ARG  FCALL))) 

(RESPOND  T 

( FCALL  I  TIRST-ARG  FCALL) 

( JUST  I CRF ATE  ' | ( APPEND  <X>  NIL)  ->  <X>|  NIL))) 

(T  (RESPOND  NIL  FCALL  (JUSTICREATE  '|no  patterns  matched. |  NIL))))) 

Tabu:  3-15.  A  Transformer  may  implement  several  transformations. 


Transformers  are  not  allowed  to  cal!  other  transformers,  l-ach  transformer  is  expected  to  be 
independent  of  any  other.  The  reason  for  this  is  that  confusion  would  reign  if  a  user  removed  a 
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transformer  from  a  transformation  set  (as  he  should  be  free  to  do)  which  was  called  by  another  he 
left  in.  Furthermore,  the  function  XFORM,  which  calls  a  transformer  and  receives  its  response,  is 
dependent  on  the  fact  that  no  other  transformer  is  called  in  the  meantime. 


3.4.2.  Selecting  a  Transformation  Set 

For  the  most  part,  selecting  the  proper  transformation  set  in  which  to  put  a  transformer 
should  be  a  simple  task.  A  transformer  which  replaces  an  AND  expression  of  no  arguments  with 
the  atom  T  should  of  course  be  put  in  the  »AND*  set,  a  transformer  to  simplify  the  expression 
(APPEND  F00  NIL)  clearly  belongs  in  the  *F CALL*  scl 

ITicrc  arc  a  few  subtleties  in  the  decision  brought  on  by  die  fact  that  some  structures  betray 
their  context  by  their  type.  The  type  name  ‘CLAUSE *.  for  example,  gives  away  the  fact  that  it 
represents  a  subtree  found  only  as  an  argument  to  a  *COND*.  A  ‘CLAUSE*  structure  will  always 
have  a  »COND*  structure  as  a  father.  Should  wc  allow  transformations  to  determine  the  context  of 
a  structure  by  checking  its  type?  Consider  the  follow  ing  transformation  rule: 

(1)  (COND  <clauses-l>  (T  (COND  <c 1 auses-2>) ) ) 

=>  (COND  <clauses-l>  <clauses-2>) 

The  final  clause  of  the  outer  COND  is  being  replaced  by  the  clauses  of  the  inner  COND.  That  is. 
one  ‘CLAUSE*  subtree  is  being  replaced  by  many.  Should  the  transformer  which  implements  this 
rule  be  a  member  of  the  *COND*  transformation  set  or  could  wc  include  the  transformer  in  the 
•CLAUSE*  set  by  rewriting  it? 

(T  (COND  <clauses>))  =>  <clauses> 

Before  I  answer  the  question,  consider  another  similar  example: 

(2)  (APPEND  <X>  (APPEND  <Y>  <Z> ) )  =>  (APPEND  <X>  <Y>  <Z>) 

If  we  put  the  first  transformer  in  the  ‘CLAUSE*  transformation  set  shouldn't  wc  also  rewrite  this 
transformer  as: 

(APPEND  <Y>  <Z>)  =>  <Y>  <Z> 

and  put  it  in  the  *FCALL*  transformation  set?  Obviously  we  can’t,  since  the  fact  that 
(APPEND  <Y>  <Z>)  is  of  type  *FCALL»  docs  not  tell  us  its  context;  we  have  no  way  of  know¬ 
ing  whether  it  appears  as  the  argument  to  another  APPEND  or  not. 

A  second  reason  for  keeping  the  first  transformer  (and  others  like  it)  in  the  higher  level 
•COND*  transformation  set  is  for  the  sake  of  the  person  who  writes  the  transformations.  Why 
should  that  person  need  to  be  aware  that  non-cvaluuble  structure  types  exist?  I  have  therefore 
decided  that  transformations  which  require  a  specific  context  should  appear  in  the  set  correspond¬ 
ing  to  that  level  of  context. 
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3.4.3.  Interfacing  to  the  Control  Functions 

Transformers  hook  into  the  control  functions  by  means  of  transformation  sets.  Recall  from 
section  3.3. 1.3  that  we  have  a  different  set  of  transformations  for  each  structure  type.  ITiough 
it  would  have  been  simple  to  have  each  set  be  a  1.1SP  list  of  the  names  of  the  appropriate  trans¬ 
formers  (and  then  map  over  that  list),  l  saw  no  reason  why  I  shouldn’t  just  implement  each  set 
as  an  implied  PROGN.  'Thus,  the  function  definitions  shown  in  Table  3-7  arc  actually  transforma¬ 
tion  sets.  A  transformer  may  be  added  or  removed  simply  by  inserting  or  deleting  the  call  from 
the  set,  remembering  that  the  order  in  which  the  calls  occur  in  a  set  is  the  order  in  which  those 
transformers  will  be  applied. 

Notice  that  all  transformers  arc  called  via  the  function  XFORM.  and  all  return  information  via 
the  function  RESPOND.  Currently  these  two  functions  do  nothing  more  than  a  FUNCALL  and 
return  a  LIST  of  their  arguments,  respectively  (and  then  upon  receipt  of  the  response,  XFORM 
causes  the  result  to  be  substituted  for  the  original  code  by  side-effecting  the  internal  form),  but 
in  the  future  we  will  cause  more  complex  things  to  happen.  In  any  case,  1  here  establish  the 
convention  that: 

(1)  No  transformer  may  be  called  directly,  but  via  XFORM  only. 

(2)  No  transformer  may  respond  directly,  but  via  RESPOND  only. 

The  significance  of  this  is  that  a  transformation  which  iterates  w  ithin  itself  must  break  one  of 
two  rules.  If  it  calls  itself  again  via  XFORM,  it  is  breaking  the  rule  that  no  transformer  may  call 
another  (remember  that  XFORM  is  dependent  on  the  fact  that  no  transformer  is  called  between  the 
time  XFORM  calls  one  and  receives  its  response).  But  if  it  calls  itself  without  the  use  of  XFORM,  it 
breaks  the  rule  stated  above.  The  solution  is  in  the  way  we  view  the  problem.  I  shall  define  a  trans¬ 
formation  which  is  implemented  via  an  iterating  transformer  to  be  incomplete  until  the  iteration 
ends.  Thus.  while  a  transformer  may  call  itself  directly,  it  docs  so  only  w  ith  the  understanding  that 
when  it  finishes,  one  transformation  has  been  applied,  regardless  of  the  number  of  iterations  the 
transformer  made. 

To  call  a  tranformcr  which  requires  global  information,  the  programmer  simply  includes  in 
the  call  as  additional  arguments  the  environment  and  the  index  of  bindings.  These  are  usually 
bound  to  ENVIRON  and  FTLIST,  respectively,  within  each  transformation  set.  Compare  the  calls 
to  AND1TRIM  and  AND1SIMPLIFY  in  the  transformation  set  for  the  *AND*  expression  structure 
type  given  in  Table  3-7. 
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There  are  a  number  of  cases  which  arise  when  writing  transformations  in  which  changes  must 
be  made  only  to  a  subtree  of  the  input  expression,  even  though  the  entire  expression  was  needed  in 
order  to  determine  whether  the  transformation  should  apply.  Consider  the  transformation  in  which 
actual  arguments  are  substituted  for  dummy  arguments  in  a  lambda  expression.  "Hie  code  to  be 
transformed  might  look  like: 

((LAMBDA  (F00  BAR) 

(COND  ((NULL  F00)  BAR) 

(T  (CONS  F00  BAR)))) 

(QUUX  S)  S) 

The  transformer  decides  after  looking  at  the  complexity  of  the  actual  arguments  ((QUUX  S)  and 
S)  and  the  use  of  the  dummy  arguments  in  the  lambda  expression,  that  S  should  be  substituted  for 
every  occurrence  of  BAR.  'Hie  resultant  expression  to  be  returned  will  appear  as: 

((LAMBDA  (F00) 

(COND  ((NULL  F00)  S) 

(T  (CONS  F00  S)))) 

(QUUX  S)) 

Only  the  following  expressions  arc  affected: 

•  the  top  level  *FCALL*  expression  which  now  calls  a  function  of  only  one  argument. 

•  each  of  the  *ATOM*  structures  representing  an  occurrence  of  BAR,  both  in  the  dummy  argu¬ 
ment  list  of  the  lambda  expression  and  in  its  body. 

•  the  *ATOM*  structure  representing  the  original  occurrence  of  S  as  one  of  the  arguments  to  the 
top  level  *FCALL*. 

We  wish  to  localize  the  transformation  so  that  only  those  sub-expressions  affected  by  the  transfor¬ 
mation  are  transformed.  T  hus,  1  have  invented  the  notion  of  slave  transformations,  which  may 
be  called  by  a  transformer  to  change  some  subtree  of  the  input  expression.  Slave  transformers 
have  their  own  XFORM-SLAVE  and  SLAVE-RESPOND  functions  which  operate  much  the  same 
as  their  counterparts,  with  the  exception  that  the  successful  application  of  a  slave  transformer  does 
not  increment  the  clock.  This  allows  the  above  described  transformation  to  appear  to  happen  all  at 
once  as  far  as  transformation  time  is  concerned;  that  is.  it  constitutes  one  transformation. 

Another  use  for  slave  transformers  is  for  the  recording  of  success  or  failure  for  transformers 
which  iterate.  Recall  from  section  3.4.3  that  tire  application  of  an  iterating  transformer  is  said  to 
constitute  only  one  transformation.  That  means  that  regardless  of  the  number  of  iterations,  only 
one  history  may  be  updated  with  success  or  failure.  Slave  transformers  allow  success  or  failure  to 
be  recorded  for  each  iteration.  This  use  (and  perhaps  other  uses  as  well)  of  slave  transformers  may 
result  in  no  change  being  made  to  the  input  expression  other  than  transformations  upon  its  sub¬ 
expressions. 
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A  slave  transformer  should  be  used  to  perform  the  following  transformation: 

(COND  <c1auses-l>  (T  (COND  <clauses-2>))) 

=>  (COND  <c1auses-l>  <clauses-2>) 

The  final  clause  of  the  outer  conditional  is  being  replaced  by  the  clauses  of  the  inner  conditional. 
That  is,  one  *CLAUSE*  structure  is  being  replaced  by  many.  As  discussed  in  section  3.4.2,  this 
transformer  appears  in  tine  *C0ND  *  transformation  set.  Rather  than  create  and  return  a  new  list  of 
the  correct  clauses,  the  transformer  should  call  a  slave  transformer  on  tine  final  clause  of  the  outer 
conditional  above  which  returns  the  list  of  *CLAUSE*  structures  <clauses-2>.  This  strategy 
assures  that  transformations  occur  as  locally  as  possible,  so  that  expressions  which  arc  not  changed 
remain  untouched  by  the  system  (a  principle  we  shall  understand  better  in  the  next  chapter). 

As  another  example  of  the  locality  of  transformations,  consider  the  rule: 

(COND  <c1auses-l>  (NIL  <exitform>)  <clauses-2>) 

=>  (COND  <clauses-l>  <clauses-2>) 

This  transformation  deletes  a  clause  which  will  never  be  evaluated,  because  its  predicate  is  known 
to  be  false.  No  other  clause  is  affected  by  the  transformation.  The  transformer  which  imple¬ 
ments  this  transformation  should  appear  in  the  *COND*  transformation  set  because  it  requires 
the  specific  context  of  a  conditional  expression  to  apply.  In  order  to  preserve  tire  locality  of  the 
transformation,  the  transformer  calls  a  slave  transformer  on  the  clause  w  ith  the  false  predicate.  The 
slave  transformer  returns  a  "nothing”  flag,  causing  the  deletion  of  the  clause  from  the  parse  tree  of 
the  conditional  expression. 

3.5.  Summary 

I  have  kept  the  transformational  component  of  the  system  as  simple  as  possible.  It  is  written  in 
I.ISP,  and  is  able  to  control  the  transformation  of  applicative  LISP  code  only.  SEXPR !  EXPAND  is 
the  driver  control  function  for  the  system,  and  will  accept  as  input  any  evaluable  LISP  expression, 
along  with  the  current  environment  and  list  of  bindings  (both  of  which  are  usually  empty  at  the 
very  beginning).  This  function  parses  the  expression  and  hands  it  over  to  the  specific  control 
function  for  its  structure  type.  The  expression  is  then  transformed  in  the  same  order  as  it  would 
be  evaluated  by  a  LISP  interpreter:  an  ordered  set  of  transformations  is  applied  to  the  top  level  of 
the  expression  only  after  its  parts  have  been  transformed.  Whenever  a  conditional  expression,  AND, 
OR,  or  an  ASSERT  is  encountered,  the  environment  is  updated;  whenever  an  internal  lambda 
function  is  encountered,  the  index  of  bindings  is  updated.  These  two  sources  of  information  are 
maintained  as  a  service  to  allow  global  transformations  to  apply.  All  transformation  occurs  via  the 
function  XFORM  only,  and  transformations  return  information  via  the  function  RESPOND  only. 
Slave  transformations  should  be  used  to  maintain  the  local  transformation  of  LISP  structures. 
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Maying  a  COMPLETE  implementation  of  the  transformational  component  we  can  now 
implement  an  accountable  component  to  accompany  it.  1  purposely  discussed  the  design 
of  an  accountable  system  before  the  implementation  of  the  transformational  component  so  that  1 
could  not  be  accused  of  limiting  my  plans  for  the  system.  Many  of  the  implementation  decisions 
made  in  the  last  chapter  in  the  name  of  simplicity  were  made  not  to  avoid  issues  in  the  implemen¬ 
tation  of  an  accompanying  accountable  component,  but  to  save  time  and  effort  in  implementing 
the  transformational  component.  As  will  be  demonstrated,  the  accountable  component  is  largely 
independent  of  the  implementation  of  the  transformational  component. 

The  issues  we  must  resolve  in  the  remainder  of  the  thesis  include: 

•  acquisition  of  information, 

•  storage  of  information,  and 
•  retrieval  of  information. 

The  first  two  points  fall  under  the  jurisdiction  of  the  recording  clement.  In  particular,  after 
reviewing  the  nature  of  the  information  we  have  decided  to  collect  as  discussed  in  section  2.4,  we 
must  rewrite  the  definitions  of  the  functions  XFORM  and  RESPOND  to  obtain  the  information. 
Then  we  will  discuss  a  means  of  storing  tlus  information  by  augmenting  the  internal  form  of  the 
argument  program. 

The  last  item  on  the  agenda  is  the  responsibility  of  the  query  clement.  Here  we  must  finally 
retrieve  the  information  at  the  command  of  the  user.  We  will  sec  that  merely  offering  to  “replay" 
specified  sections  of  the  transformation  process  can  be  fairly  useful,  though  trying  to  outguess  the 
user  by  providing  the  answers  to  more  general  questions  might  be  more  inlercsting. 
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4.1.  Acquisition  of  Information 

Recall  from  section  2.5.1  our  summarization  of  the  recording  element’s  task: 

•  Transformations  must  include  in  their  description  a  set  of  input  and  output  assertions,  available 
upon  request. 

The  motivation  for  this  requirement  was  to  couple  this  information  with  the  truths  for  a 
particular  piece  of  code  in  order  to  determine  why  a  transformation  failed  or  succeeded.  For 
example,  if  a  transformation  T  requires  properties  a,  b,  and  c,  but  only  a  and  b  are  true  at 
some  point,  we  may  assume  that  T  failed  because  either  c  was  false  or  unknown. 

Since  we  have  chosen  to  implement  transformations  as  procedures,  they  can  be  made  to 
return  the  reason  they  failed  or  succeeded  themselves.  We  will  therefore  let  the  procedure 
definition  stand  as  the  description  of  input  and  output  assertions  for  a  transformation,  and 
require  that  transformers  return  justifications  for  their  actions. 

•  Ikforc  any  transformations  apply,  a  set  of  truths  for  each  piece  of  code  (dependent  on  its 
context)  must  be  collected  and  recorded  along  with  their  Justifications.  That  is,  before  any 
transformation  applies  to  a  particular  piece  of  code ,  truths  for  that  code  must  be  recorded. 

I  nis  is  the  environment,  and  we  have  discussed  it  in  the  last  chapter.  We  do  need  to  add  the 
justifications:  we  will  require  that  ENV!  RECORD-TEST  be  given  a  justification  for  the  truth  it 
is  to  record,  and  record  that  justification  along  with  the  test.  If  ENV!  RECORD-TEST  docs  any 
fancy  logic  it  should  augment  the  justification  it  is  given  with  a  note  to  that  effect.  Similarly, 
ENV!  AOD-TEST  should  justify  any  truth  it  adds  to  the  environment. 

Though  the  environment  and  its  index  of  bindings  could  have  been  stored  as  part  of  the 
internal  form  of  the  program  during  the  implementation  of  the  transformational  component, 
they  were  instead  created  and  maintained  as  separate  variables  which  were  passed  from  control 
function  to  control  function,  and  finally  to  die  transformation  set  to  be  used  by  the  transformers. 
1'hc  accountable  component,  however,  must  record  the  environment  and  index  used  for  the 
transformation  of  each  expression  so  that  its  information  is  not  lost  once  the  transformation  set 
is  exited,  as  is  the  case  now.  It  will  do  this  immediately  after  a  transformation  set  is  entered. 

•  After  each  transformation  is  attempted,  it  must  provide  to  the  recording  clement  the  following 
information: 

(1)  what  code  it  attempted  to  transform; 

(2)  whether  or  not  it  was  successful; 

(.1)  if  it  was  successful,  the  resultant  code. 

Since  all  transformations  arc  called  via  the  function  XFORM,  and  XFORM  is  responded  to  via 

RESPOND.  XFORM  must  already  know  (1).  since  XFORM  obviously  passes  the  transformation  the 
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: ; ; T  ransformer 

(DEFUN  ORISIMPLIFY  (ORTST) 

(COND  ( (ARGSINULL?  (ORIARGLIST  ORTST)) 

(RESPOND  T  ( SEXPR ! COPY  FALSE)  (JUST 'CREATE  '  | ( OR )  =>  NIL  |  NIL))) 

( ( ARGSIONE?  (ORIARGLIST  ORTST)) 

(RESPOND  T 

(ARGSIARG  (ORIARGLIST  ORTST)) 

(JUSTICREATE  '|(OR  <*>)  *>  <*>l  NIL))) 

(T  (RESPOND  NIL  ORTST  (JUSTICREATE  ' | no  patterns  match. |  NIL))))) 

; ; ;Transformer 

(DEFUN  FCALL I S IMP-NOT  (NULLTST) 

(CONO  ( ( NOTEXPR?  NULLTST) 

(LET  ((LOGARG  (ARGSIARG  ( FCALL I ARGLIST  NULLTST)))) 

(CONO  ((NULL?  LOGARG) 

(RESPOND  T 

(SEXPRICOPY  TRUE) 

(JUSTICREATE  '  | ( NOT  NIL)  =>  T|  LOGARG))) 
((TRUE?  LOGARG) 

(RESPOND  T 

(SEXPRICOPY  FALSE) 

(JUSTICREATE  '|(NOT  T)  *>  NIL |  LOGARG))) 

(T  (RESPOND  NIL 

NULLTST 

(JUSTICREATE 

‘I  the  arg  to  NOT  or  NULL  is  unknown. | 
NIL)))))) 

(T  (RESPOND  NIL  NULLTST  (JUSTICREATE  '|no  patterns  match. |  NIL))))) 
Table  4-1.  Transformations  must  return  justifications. 


(DEFUN  RESPOND  (APPLIED?  OBJECT  JUST) 

(CONO  ((NOT  APPLIED?)  (LIST  ••RESPONSE.*  ••FAILED.*  JUST)) 
(T  (LIST  ••RESPONSE**  OBJECT  JUST)))) 

Table 4-2.  New  definition  of  RESPOND. 


code  to  transform.  So  there  is  no  need  for  a  transformation  to  explicitly  return  that  information. 
Currently  a  transformer  already  supplies  the  information  required  by  (2)  and  (3):  if  it  fails  it 
executes  the  form  ( RESPOND  NIL  <input  express  ion> ).  and  if  it  succeeds,  it  executes  the 
form  (RESPOND  T  Cresultant  express  ion>). 

Transformations  arc  required  to  return  a  three-valued  response,  then:  a  flag  signifying 
whether  or  not  they  succeeded  (T  or  NIL.  respectively),  the  resultant  expression,  and  a  justification 
for  the  success  or  failure.  The  new  definition  of  RE  SPOND  may  he  seen  in  Table  4-2. 
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4.1.1.  Justifications 

A  justification  consists  of  an  explanation,  and  optionally,  an  expression.  It  may  either  justify 
a  transformer  which  succeeds  or  fails,  or  justify  the  addition  of  a  piece  of  common  contextual 
information  to  the  environment.  In  the  first  case,  its  explanation  is  a  brief  phrase  which  completes 
the  sentence  “This  expression  was  transformed  because. . .”  or  die  sentence  “The  transformer 
< transformed  failed  to  transform  this  expression  because. . .".  In  the  second  case,  its  explanation 
should  complete  the  sentence  “This  expression's  value  is  <value>  because. . If  die  explanation 
refers  to  some  other  expression  (such  as  “. .  .it  is  in  die  true  branch  of  the  conditional  clause:”), 
then  that  expression  may  be  included  in  the  justification  and  will  be  printed  when  die  justification 
is  displayed. 

The  expression  component  of  a  justification  may  also  serve  as  a  link  to  previous  justifications. 
Suppose  ENV! RECORD-TEST  is  asked  to  record  the  fact  that  (AND  S  T)  is  true  because  the 
system  is  about  to  transform  expressions  which  are  “widiin  the  true  branch  of  the  conditional 
clause"  whose  predicate  is  that  expression.  Then  ENV! RECORD-TEST  will  make  two  separate 
entries:  diat  S  is  true  and  that  T  is  true,  because  each  of  them  is  “an  argument  to  a  true  AND 
expression.”  Later,  if  a  transformation  succeeds  because  S  is  true,  it  may  return  the  jusdfication 
associated  with  S  as  its  justification.  The  justification  of  S  as  recorded  by  FNV! RECORD-TEST  is 
"an  argument  to  a  true  AND  expression”;  there  is  no  clue  left  as  to  wTiv  the  AND  expression  is  true! 
1  solved  this  problem  in  die  following  manner: 

•  ENV!  RECORD-TEST  will  now  record  the  predicate  and  justification  exactly  as  it  receives  them 
in  addition  to  recording  any  pieces  of  the  predicate.  This  leaves  a  record  as  to  why  die  AND 
expression  is  true  in  the  above  example. 

•  When  pieces  of  a  predicate  arc  recorded,  they  are  given  a  justification  which  consists  of  an 
explanation  specifying  the  nature  of  die  predicate  piece  ("argument  of  a  true  NOT  expression”, 
"argument  of  a  false  OR  expression”,  etc),  and  the  expression  representing  die  predicate  the 
piece  came  from. 

This  will  allow  the  query  element  to  perform  some  simple  dependency  “backlooking”:  When  a 
user  wishes  to  know  why  S  is  true,  it  can  print  die  explanation  component  of  the  justification 
of  S,  followed  by  the  expression  component,  then  the  justification  recorded  on  die  environment 
for  (hat,  and  so  on.  until  die  originally  asserted  expression  is  found.  The  new  definition  of 
ENV!  RECORD-TEST  is  shown  in  Table 4-3. 

The  function  XFORM  now  has  all  the  information  relating  to  the  transformation  of  an  expres¬ 
sion.  and  only  needs  to  store  that  information  so  that  (1)  future  transformations  will  sec  the  correct 
expression,  and  (2)  the  query  clement  can  obtain  any  intermediate  version  of  die  expression  it 
desires  At  the  same  time,  die  recording  element  must  represent  this  information  “in  an  efficient 
and  accessible  manner.” 
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(DEFUN  ENV! RECORD-TEST  (ENVIRON  1ST  OUTCOME  JUST  FTL  1ST ) 

(COND 

((NULL  TST )  ENVIRON) 

((ATOM?  TST) 

(ENVIADD-TEST  ENVIRON 

(ENV-EUCREATE  (  F  AL  1ST  !  SUBL  IS  FTLIST  TST)  OUTCOME  JUST))) 

( ( ANDEXPR?  TST) 

(COND  (OUTCOME  ;  recording  a  true  AND  test 

( ENV! RECORD-TESTS 

«  (ENV! ADD-TEST  ENVIRON 

•  (ENV-EUCREATE  (  F  AL  1ST !  SUBL  IS  FTLIST  TST)  T  JUST)) 
(ANDiARGLIST  TST) 

T 

•  (JUSTiCREATE  '(Argument  to  a  true  AND  expr.|  TST) 

FTLIST)) 

(T  ( ENV ! ADD-TEST  ;  recording  a  false  AND  test 
ENVIRON 
( ENV-EL ICREATE 
( FALIST ! SU8LIS 
FTLIST 
(LET 

((REFINED 

{ ARGS ! REFINE  ( AND! ARGL 1ST  TST)  OUTCOME  ENVIRON  FTLIST))) 
(COND  ( ( ARGS ! ONE?  REFINED)  (ARGS ! ARG  REFINED)) 

(T  (ANDICREATE  REFINED))))) 

OUTCOME 

•  JUST))))) 

( (OREXPR?  TST) 

( COND  (OUTCOME  ;  recording  a  true  OR  test 

( ENV! ADD-TEST 
ENVIRON 
( ENV-EL ! CREATE 
(FALIST ! SUBLIS 
FTLIST 
(LET 

((REFINED 

(ARGS! REFINE  (OR! ARGLIST  TST)  OUTCOME  ENVIRON  FTLIST))) 
(COND  ( ( ARGS ! ONE?  REFINED)  (ARGS! ARG  REFINED)) 

(T  (ORiCREATE  REFINED))))) 

OUTCOME 

•  JUST))) 

(T  ;  recording  a  false  OR  test 

(ENV! RECORD-TESTS 

•  ( ENV! ADD-TEST  ENVIRON 

•  (ENV-EIUCREATE  (  FALIST  !  SUBL  IS  FTLIST  TST)  NIL  JUST)) 
(OR! ARGLIST  TST) 

OUTCOME 

•  (JUSTiCREATE  ' (Argument  to  a  false  OR  expr. |  TST) 

FTLIST)))) 

( ( NOTEXPR?  TST)  ;  strip  off  NOT  and  toggle  outcome 

(ENVIRECORO-TEST 

•  ( I NV! ADD-TEST  ENVIRON 

.  ■  (ENV-EL ! CREATE  ( FALIST ! SUBLIS  FTLIST  TST)  OUTCOME  JUST)) 

( ARGS ! ARG  ( FCALL ! ARGL 1ST  TST)) 

(NO!  OUTCOME) 

•  (JUSTICREATE  '(Argument  to  a  NOT  expression.!  TST) 

FTLIST)) 

(T  (ENV! ADD-TEST  ENVIRON  ( ENV-EL !CRE ATE 

(FALIST! SUBL I S  TTLIST  TST)  OU.COME  JUST))))) 


Ta hi. i  4-1.  The  environment  must  now  maintain  justifications  for  its  entries. 


62 


Chapter  Four 


The  account  able  Component 


4.2.  Storage  of  Information 

Before  we  consider  a  means  of  storage  for  the  information  in  hand,  let's  think  quickly  about 
the  nature  of  its  retrieval.  A  programmer  who  submits  his  code  to  the  accountable  system  will  be 
interested  in  observing  its  effect  on  his  program.  Thus,  he  will  ask  questions  about  specific  sections 
of  code.  One  w  ho  has  w  ritten  a  new  transformation  may  test  it  with  a  random  program,  and  then 
ask  questions  pertaining  to  the  actions  of  that  transformation.  And  someone  who  is  concerned 
with  the  order  of  transformation  application  will  ask  questions  about  what  happened  at  wT: .  .  mr. 
Ihcre  arc  then  three  possible  indicics:  the  code  transformed,  the  name  of  the  transformer,  or  he 
time  of  transformation.  We  w  ill  use  them  all. 


4.2.1.  By  Code 

If  the  transformational  component  of  the  system  were  to  be  run  without  any  accountable 
capabilities,  it  could  simply  replace  any  subtree  of  the  parse  tree  with  the  new  one  whenever  a 
transformation  applied.  But  the  accountable  component  w  ill  remember  c\ cry  intermediate  version 
of  the  program.  If  there  are  fifty  different  versions  (because  there  were  fifty  successful  transforma¬ 
tion  applications),  storing  fifty  different  parse  trees  is  clearly  not  efficient,  whether  the  program  is 
large  or  small.  Instead,  I  have  chosen  to  augment  the  internal  form  of  the  parse  tree  by  allowing 
multiple  v  ersions  of  each  subtree.  When  a  transformation  applies,  the  resulting  subtree  is  added  to 
the  parse  tree  as  version  n-f-  1.  We  can  write  a  special  printer  which,  given  the  entire  tree  and  a 
version  number,  will  print  the  appropriate  subtree  when  it  arrive'  at  a  node  in  die  parse  tree  which 
has  more  than  one.  1'his  printer  really  constitutes  a  translator  or  conversion  routine  between  the 
internal  form  and  the  executable  source  code. 

At  each  node  of  the  parse  tree  is  a  structure  called  a  CELL.  Cells  contain  (as  well  as  other 
information  which  we  will  mention  later)  a  list  of  die  different  subtrees  for  this  node:  initially  this 
list  contains  only  one  subtree.  Hach  subtree  is  represented  by  a  structure  called  a  MONK,  livery 
successful  transformation  causes  a  MONK  to  be  created.  Because  we  want  to  record  unsuccessful 
transformations  as  well,  we  w ill  create  a  structure  called  a  HERMIT  w henever  one  fails.  Besides  the 
subtree  it  represents,  each  MONK  contains  a  version  number,  the  name  of  the  transformer  which 
created  it,  and  the  justification  for  that  transformation,  fiach  HE  RMI T  contains  the  version  number 
of  the  subtree  which  wasn't  transformed,  the  name  of  the  transformer  which  didn't  transform  iL 
and  a  justification  for  the  failure. 


§4.2.3 
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(Of fUN  XFORMER I RECORD  ( X FORMER  CELL  VERSION) 

(PUTPROP  XEORMER  (CONS  (CONS  VERSION  CELL) 

(GET  XEORMER  'APPLIED))  'APPLIED)) 

(DEE UN  XEORMER ! RECORDE  (XEORMER  CELL  VERSION) 

(LET  ((FAILS  (GEE  XEORMER  'FAILED))) 

(COND  ((NULL  FAILS) 

( PUI PROP  XEORMER  (LIST  (CONS  VERSION  CELL))  'TAILED)) 

((>  (CAAR  FAILS)  VERSION) 

(ERROR  '(Fails  out  of  order. |  XEORMER)) 

((<  (CAAR  FAILS)  VERSION) 

(PUTPROP  XEORMER  (CONS  (CONS  VERSION  CELL)  FAILS)  'FAILED)) 
((MEMO  CELL  ( CDAR  FAILS))  NIL) 

(T  (PUTPROP  XEORMER 

(CONS  (CONS  VERSION  (CONS  CELL  (CDAR  FAILS))) 

( CDR  FAILS)) 

'FAILED))))) 

Table  4-4.  Functions  which  record  information  by  transformer  name. 


4.2.2.  By  Transformer 

If  someone  were  intercsied  in  the  activities  of  transformer  T,  pointing  them  to  each  CELL 
transformed  by  T  and  giv  ing  them  the  time  of  each  transformation  would  be  enough  to  allow 
them  to  then  ask  for  any  other  information  that  might  be  stored  in  the  cells.  X  FORME  It!  RECORD 
takes  the  name  of  a  transformer,  die  argument  cell,  and  the  current  time,  and  records  the  fact  that 
that  transformer  affected  the  said  cell  at  die  given  time.  XFORMER !  RECORDF  records  the  same 
information  for  transformations  that  fail.  (The  definitions  of  these  two  functions  arc  given  in  Table 
4-4.)  Note  that  this  is  a  record  of  whether  or  not  a  transformer  succeeds.  Recall  that  a  transformer 
can  implement  more  than  one  transformation  rule.  If  any  rule  succeeds,  the  transformer  is  said  to 
succeed.  Only  if  all  the  rules  implemented  by  a  transformer  fail  docs  die  transformer  fail. 


4.2.3.  By  Version 

We  can  index  the  information  by  version1  in  much  the  same  way  as  by  transformer. 
CLOAKRACK !  HANG -UP  maintains  an  association  list  of  transformation  times  and  cells  transformed 
at  those  times  (called  the  “cloakrack”,  because  a  monk  hangs  up  its  cloak  before  entering  the 
cell).  RA'GRACK !  HANG-UP  maintains  a  similar  association  list  for  transformations  which  fail.  Its 
definition  differs  in  that  many  transformations  may  fail  during  the  same  transformation  time  since 
the  dock  is  not  incremented  until  a  transformation  succeeds.  1  claim  that  it  is  relatively  unimpor¬ 
tant  to  keep  track  of  which  transformation  fails  before  another,  because  no  subtree  changes.  If 


!  I  will  use  the  terms  "version”  and  "transfomiaiion  lime"  interchangeably.  since  version  n  of  a  program  is  that 
parse  tree  of  the  program  which  was  created  ai  transformation  time  n.  wnh  one  exception:  version  0  is  ciealed  not 
by  transformation  but  by  comcrsion  to  the  internal  Form 
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(DfFUN  CLOAKRACKIHANG-UP  (CELL  VERSION) 

( S£ 7 0  CLOAKRACK  (CONS  (LIST  VERSION  CELL)  CLOAKRACK))) 

(OEEUN  RAGRACK ! HANG -UP  (CELL  VERSION) 

(CONE)  ((NULL  RAGRACK)  ( SE  T  0  RAGRACK  (LIST  (CONS  VERSION  CELL)))) 

(I  (LEI  ((V  (CAAR  RAGRACK))) 

(COND 

((>  V  VERSION) 

(ERROR  '|RagracK  out  of  order. |)) 

((<  V  VERSION) 

(SI  10  RAGRACK  (CONS  (LIST  VERSION  CELL)  RAGRACK))) 

(  (Ml  MQ  CELL  (CI)AR  RAGRACK))  NIL) 

(I  ( SE  TO  RAGRACK  (CONS  (CONS  V  (CONS  CELL  ( CDAR  RAGRACK))) 
(COR  RAGRACK))))))))) 

Tabu:  4-5.  Definitions  of  functions  which  record  information  by  version. 


(DEFUN  XFORM  (XIORMER  CELL  ARGS) 

(LEI  ((RESPONSE  (APPLY  XFORMER  (CONS  CELL  ARGS) ) > ) 

(COND  ((APPLIED?  RESPONSE) 

(CELL ISTORE-MONK  CELL  (  RESPONSE ! HACK  RESPONSE  CELL  XFORMER)) 
( XFORMER ! RECORD  XFORMER  CELL  X 1 0RMI1 ME ) 

( CLOAKRACK IEIANG-UP  CELL  XFORMTIME) 

( XFORM! IME ! INCR ) ) 

(T  ( CELL ! STORE -HERMIT  CELL  ( RESPONSE ! MACKE  RESPONSE  XFORMER)) 
(  Xf  OHITE  R  !  RECORDF  XFORMER  CELL  XFORMI  IME  ) 

( PAGR.ACK!  JIANG -UP  CELL  XFORMTIME))))) 

Table. 4-6.  Final  definition  of  XFORM. 


transformers  7-1  and  T-2  both  attempt  to  transform  expression  E  and  both  fail,  it  is  immaterial 
which  failed  first.  Kxprcssion  E  remains  the  same  before,  after,  and  during  both  attempts.  The 
defmitionsof  CLOAKRACK !  HANG-UP  and  RAGRACK  !  HANG-UP  are  shown  in  Table  4-5. 


4.2.4.  The  Final  Version  of  XFORM 

We  can  now  complete  our  definition  of  XFORM  by  adding  calls  to  functions  which  will  store 
the  collected  information,  look  at  T  able  4-6.  RESPONSE  !  HACK  takes  the  response,  the  cell  which 
was  transformed,  and  the  name  of  the  transformer,  and  creates  a  MONK  which  is  then  stored  in 
the  transformed  cell.  After  creating  cross-references  to  this  information  by  transformer  and  trans¬ 
formation  time,  die  “clock"  is  incremented  in  order  to  be  ready  to  pro\idc  the  time  (version) 
for  the  next  transformation.  If  a  transformat  ii  in  fails,  first  RESPONSE  IHACKF  creates  a  HERMIT 
with  information  pertaining  to  the  attempt,  and  returns  the  HERMIT  lor  storage  in  the  cell.  Then 
the  information  is  again  cross-ivfcronccd  by  transformer  and  Uunsformulion  time,  but  the  clock  is 
'//>/  incremented.  T  he  definitions  of  RESPONSE  IHACK  and  RESPONSE  IHACKF  will  be  discussed 


(DEFUN  SFXPR.'CONVERT-IO-X  (SEXPR) 

(COND  ((CEU?  SEXPR)  SEXPR) 

((ATOM?  SEXPR)  ( ATOM ! CONV! RT -TO-X  SEXPR)) 

((DEEETTE?  SEXPR)  ( DE  F  ETIL  !  CONVERT -TO- X  SEXPR)) 

((QUOTE?  SEXPR)  (QUOTE ! CONVERT -TO-X  SEXPR)) 

((COND?  SEXPR)  ( COHO ! CONVE RT -TO- X  SEXPR)) 

((ANOEXPR?  SEXPR)  ( AND ! CONVERT -TO- X  SEXPR)) 

( (OREXPR?  SEXPR)  ( OR ! CONVERT -TO- X  SEXPR)) 

(T  ( f CALL ! CONVERT -TO- X  SEXPR)))) 

(DEfUN  ARGSICONVERT-TO-X  (ARGLIST) 

(COND  ((NULL  ARGLIST)  (LIST  ( NOTH1 NG ! CREATE )) ) 

((CELL?  (ARGSIARG  ARGLIST))  ARGLIST) 

(T  (MAPCAR  (FUNCTION  SEXPR ! CONVERT -TO-X )  ARGLIST)))) 

(DEFUN  ATOM ! CONVERT -TO-X  (AT) 

(COND  ((CELL?  AT)  AT) 

(T  (CELL ! CREATE  AT  ..ATOM*.  ' ATOM ! CONVERT -TO-X ))) ) 

(DEFUN  BINDINGICONVERT-TO-X  (DARGS) 

(COND  ((CELL?  DARGS)  DARGS) 

(T  ( B INOING ! CREATE  ( ARGS ! CONVERT -TO- X  DARGS))))) 

(DEFUN  DEEETTE! CONVERT -TO-X  (DEF) 

(COND  ((CELL?  DEF)  DEF) 

(T  (DEFETTE 'CREATE  ( ATOM! CONVE RT -TO-X  (DE FETTE ! NAME  DEF)) 

(BINDING! CONVERT -TO-X  ( DEFE T T E ! B HIDING  DEF)) 
(SEXPR ! CONVERT -TO-X  (DEFETTE ! BODY  DEF)))))) 


Tabu:  4-7.  Functions  for  convening  to  the  internal  form. 


4.3.  A  Look  at  Cells 

A  cell  contains  the  “history”  of  a  subtree  of  a  parse  tree.  It  charts  the  transformation  of  that 
subtree  via  its  list  of  monks,  each  of  which  points  to  a  different  version.  The  subtree  itself  con¬ 
stitutes  a  parse  tree  though,  so  each  of  its  subtrees  arc  kept  track  of  by  cells  and  monks.  The  use  of 
cells  as  the  internal  form  of  the  argument  program  raises  a  number  of  issues  regarding  conversion 
to  and  the  transformation  of  that  form. 

4.3.1.  Conversion  to  the  Internal  Form 

Transformations  may  operate  only  on  cells,  since  XFORM  expects  to  get  a  cell  into  which  it 
can  store  a  new  monk.  I  hus,  every  subtree  which  might  be  a  candidate  for  transformation  must  be 
converted  to  a  cell.  We  have  already  decided  which  I  .ISP  structures  to  pros  idc  w  ith  transformation 
sets  (in  section  3.3. 1.3).  We  will  represent  each  of  the  structures  whose  t\  pcs  arc  defined  by  the 
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cdr  x 

Figure 4-1.  Cell  representation  of  (CONS  (CAR  X)  (APPEND  (CDR  X)  Y)). 


transformational  component  (and  given  in  Tabic  3-5  (page  39))  as  cells  of  the  same  type.  Cells  arc 
used  not  only  to  represent  LISP  code,  but  to  contain  some  useful  information  about  that  code  as 
well. 


4.3.1. 1.  Cell  Types 

The  cell  representation  of  the  expression  (CONS  (CAR  X)  (APPEND  (CDR  X)  Y))  is 
illustrated  in  Figure  4-1.  Fach  atomic  symbol  of  llic  expression  is  a  leaf  of  the  tree,  and  may  be 
either  of  type  «AT0M*  or  *FUNCNAME*. 

What  about  the  few  remaining  pieces  of  I  .ISP  code  which  arc  not  typed?  Consider  the  dummy 
argument  list  of  a  function  definition,  or  the  special  form  headers  COND.  AND,  OR.  QUOTE,  etc.  My 
decisions  on  these  issues  were  made  as  follows: 

( 1)  The  dummy  argument  list  will  be  represented  as  a  cell  of  type  »B1  NDING*  even  though  trans¬ 
formations  are  unlikely  to  apply  to  the  cell.  Representing  the  list  as  a  cel!  will  cause  a  function 
definition  (a  *DEFETTE*  cell)  to  consist  of  a  list  of  three  cells:  *FUNCNAME«.  *BINDING*, 
and  a  body  which  may  be  any  evaluable  structure  type,  and  thus  a  cell  itself. 

(2)  Special  forms  each  have  their  own  cell  type,  and  thus  the  header  itself  need  not  be  made  a 
cell.  Instead,  each  cell's  monk  contains  a  list  of  the  arguments  to  the  special  form.  Thus,  a 
*C0ND*  cell’s  monk  contains  a  list  of  *CLAUSE*  cells,  an  *AND*  cell's  monk  contains  a  list 


t 
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(OEFUN  SEXPR ! CONVERT-FROM-X  (SEXPR  &optional  VERSION) 

(CONO 

((NOT  (CELL?  SEXPR))  SrXPR) 

(T  (LET  {(TYPE  (SEXPRI1YPE  SEXPR  VERSION))) 

(CONO 

( (EQ  ..ATOM*.  TYPE)  ( ATOM [CONVERT -FROM- X  SEXPR  VERSION)) 

( ( EO  ».f UNCNAMF..  TYPE)  ( TUNC NAME ! CONVI RT -F ROM- X  SEXPR  VERSION)) 
((EQ  ..BINDING..  TYPE)  ( B I NDI NG ! CONVER I -F ROM- X  SEXPR  VERSION)) 
((EO  ..OEFETTE..  TYPE)  ( DE FE TT E ! CONVE RT -I ROM-X  SEXPR  VERSION)) 
((EQ  ..QUOTE..  TYPE)  (QUOT E ! CONVERT  -  FROM- X  SEXPR  VERSION)) 

((EQ  ..CONO..  TYPE)  ( CONO! CONVERT -f ROM-X  SEXPR  VERSION)) 

{(EQ  ..CLAUSE*.  TYPE)  ( CL AUSE ! CONVE RT - F ROM-X  SEXPR  VERSION)) 

((EQ  ..AND..  TYPE)  ( AND ! CONVE  RT - F  ROM-X  SEXPR  VERSION)) 

((EQ  ..OR..  TYPE)  (ORICONVERT-FROM-X  SEXPR  VERSION)) 

((EQ  ..FCALL..  TYPE)  ( FCALL ! CONVERT -FROM- X  SEXPR  VERSION)) 

(T  (ERROR  ' IUnknown  type  --  SEXPR ! CONVERT-FROM-X |  SEXPR))))))) 

(DEFUN  ARGS ! CONVERT-FROM-X  (ARGLIST  .ARGLISTVERSION.) 

(DECLARE  (SPECIAL  .ARGLISTVERSION.)) 

(COND  ({ARGSINULL?  ARGLIST  .ARGLISTVERSION.)  NIL) 

((CELL?  (ARGS! ARG  ARGLIST  .ARGLISTVERSION.)) 

( SEXPR !MAPCAR 

(FUNCTION  (LAMBDA  (EXPR) 

( SEXPR ! CONVERT-FROM-X  EXPR  .ARGLISTVERSION.))) 
ARGLIST 

•ARGLISTVERSION.)) 

(T  ARGLIST))) 

(DEFUN  ATOM ! CONVERT-FROM-X  (AT  VERSION) 

(CONO  ((CELL?  AT)  (SEXPR ! TREE  AT  VERSION )) 

(T  AT))) 

(DEFUN  BINDING! CONVERT-FROM-X  (BINDING  VERSION) 

(COND  ((NOT  (CELL?  BINOING ) )  BINDING) 

(T  (BINOING! CREATE 

( ARGS ! CONVERT-FROM-X 

(BINDING ! DARGLIST  BINDING  VERSION)  VERSION))))) 

(DEFUN  DEFETTE ! CONVERT-FROM-X  (OEF  VERSION) 

(COND  ((CELL?  DEF) 

(DEFETTE ! CREATE 

(ATOM! CONVERT-FROM-X  (DEFE TTE ! NAME  DEE  VERSION)  VERSION) 

(BINOING! CONVERT-FROM-X  (OE FETTE ! B I NDING  DEF  VERSION)  VERSION) 
(SEXPR ! CONVERT -FROM-X  ( DEF ETT E ! BODY  DEF  VERSION)  VERSION))) 

(T  OEF))) 

Table;  4-8.  Functions  for  converting  from  the  internal  form. 


of  evaluable  cells,  etc.  Wc  arc  therefore  allowing  cell  types  to  take  the  place  of  actual  code. 
That  is.  COND,  AND.  OR,  etc,  will  never  appear  as  leaves  of  the  parse  tree. 

I  he  rather  pleasant  results  of  these  two  decisions  is  (hat  the  subtree  contained  in  any  cell's 
monk  is  cither  an  atom,  a  cell,  or  a  list  of  cells,  with  the  exception  of  die  .QUOTE*  cell, 
whose  monk  may  contain  any  bare  s-expression.  Whatever  that  s-expression  is,  the  conversion 
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(DEfUN  CELLICREATE  (SEXPR  TYPE  VERSION  CREATOR) 

(LET  ((ID  (INTERN  (MYGENSYM  'CELL)))) 

(LET  ((NEWCELL 

(LIST  ••CELL** 

ID 

(LIST  (BPL 1ST !CREATE ) 

(RANGE iCREATE  VERSION  VERSION) 

( BVAL !CRE ATE  NIL  NIL  NIL) 

NIL  ;  slot  for  environment 
NIL)  ;  slot  for  index  of  bindings 
(MONKICREATE  TYPE  VERSION  SEXPR  CREATOR 

(JUST ! CREATE  ' | F i rs t  monk|  NIL)  NIL)))) 

(SET  ID  NEWCELL) 

(CELL! INSTALL-BACKPOINTEES  NEWCELL)))) 

(DEFUN  MONKICREATE  (TYPE  VERSION  SEXPR  XFORMER  JUSTIFICATION  PTCPOINTER) 

(LIST  ..MONK**  TYPE  VERSION  SEXPR  XFORMER  JUSTIFICATION  PTCPOINTER)) 

Tabu- 4-9.  The  functions  which  create  cells  and  monks. 


routines  won’t  touch  it.  Table  4-7  and  Table  4-8  show  some  of  the  conversion  functions. 
SEXPR ! CONVERT-TO-X  and  SEXPR ! CONVERT -FROM-X  are  driver  functions  which  convert 
any  s-expression  to  or  from  the  internal  form,  respectively. 

The  type  of  a  cell  is  stored  in  the  monk  which  represents  it,  since  transformation  of 
a  subtree  may  result  in  its  being  of  a  different  type  than  before.  For  example,  the  rule 
(COND  (T  <X> ) )  =>  <X>  changes  the  parse  tree  of  (COND  (T  (CAR  FOO)))  from  type 
COND  to  type  FCALL.  The  type  of  a  cell  is  defined  to  be  the  type  of  the  most  recent  monk. 


4.3. 1.2.  Cell  Creation 

The  function  CELL  !  CREATE  creates  a  cell.  It  requests  the  subtree  and  type  to  be  installed 
in  the  first  monk,  and  the  name  of  the  function  requesting  the  creation.  The  first  monk  of  each 
cell  is  the  one  given  at  creation,  and  always  has  version  0.  whether  it  was  created  during  the  trans- 
formalion  process  or  during  the  initial  conversion  of  the  I  .ISP  program  to  the  internal  form.  This 
will  not  raise  ambiguities  when  converting  back  to  executable  I  .ISP  code,  since  even  though  a  cell 
may  have  a  0  version  monk,  it  can  not  be  reached  if  its  parent  cell  doesn’t  point  to  it  at  time  0. 
Consider,  for  example,  the  transformation  that  replaces  a  function  call  by  its  definition.  Suppose 
•  FUNCNAME*  cell  CELL-33  is  FOO  at  time  3.  and  at  time  4  is  replaced  by  the  lambda  expression 
(LAMBDA  (X)  (CAR  X) ).  Then  every  subccll  containing  a  subtree  of  the  lambda  expression 
will  be  at  version  0  since  the  expression  is  newly  copied  before  replacement.  Printing  CELL-33 
at  time  3  will  produce  FOO  though,  not  the  lambda  expression,  because  the  version  3  monk  of 
CELL-33  points  to  FOO. 


It  is  useful  to  be  able  to  "back-up"  to  the  father  of  a  cell,  and  to  the  father  of  that  cell,  and  so 
on.  if  desired.  Since  which  cell  is  the  father  of  another  cell  changes  as  transformations  apply,  it  is 
not  sufficient  to  maintain  a  single  static  back-pointer  for  each  cell.  Instead,  when  a  cell  is  created 
it  is  given  a  back-pointer  list,  initially  NIL.  However,  the  back-pointer  list  of  each  j  ubce!!  is  made 
to  point  to  the  newly  created  cell.  A  back-pointer  consists  of  a  cell  and  a  version  number.  To 
obtain  the  back-pointer  of  a  cell  we  use  the  function  CELL  IBP,  which  takes  a  cell  and  a  version 
number  and  returns  the  back-pointer  of  the  cell  for  that  point  in  time.  Suppose,  for  example,  that 
CELL-45  has  the  back-pointer  list: 

(  (23  .  CELL-88)  (14  .  CELL-23)  (11  .  CELL-23)  (10  .  CELL-4)  ) 

Then(CELL!BP  CELL-45  15 )  will  return  the  back-pointer  ( 14  .  CELL-23).  The  function 
BP !  CELL  will  return  the  cell  of  a  back-pointer.  Then  ( BP !  CELL  (CELLIBP  CELL-45  10)) 
will  return  CELL-4.  If  the  back-pointer  of  CELL-45  is  requested  for  some  version  less  than  10, 
CELLIBP  will  return  NIL,  indicating  that  CELL-45  had  no  father  at  that  time. 

Recording  the  back-pointers  of  a  cell  will  allow  us  to  easily  implement  the  function 
CELL  I  FATHERS.  This  function  Likes  a  cell  and  a  version  number  and  returns  a  list  of  the  cell's 
current  ancestors;  that  is,  a  list  of  all  the  cells  currently  on  the  direct  path  from  the  given  cell  to  the 
root  cell  of  the  parse  tree. 

At  creation,  each  cell  is  given  a  “range",  which  allows  us  to  determine  the  times  at  which  it 
or  any  of  its  subcclls  underwent  transformation.  A  range  consists  of  a  minumum  and  maximum 
version,  both  of  which  arc  0  at  creation.  If  a  transformation  applies  to  CELL-45  at  time  5,  the 
range’s  maximum  is  changed  to  5,  and  every  range  of  the  transformed  cell's  current  ancestors  is 
updated  as  well.  Hence,  if  in  the  end  CELL-45  has  a  range  with  minimum  5  and  maximum  49, 
we  know  that  every  transformation  which  occurred  from  time  5  to  time  49  applied  to  CELL-45  or 
one  of  its  subcells. 

Though  the  initial  monk  of  a  cell  always  has  version  0,  0  is  not  included  in  the  range.  This 
is  because  die  I. ISP  program  tree  which  is  input  to  the  transformation  system  is  converted  to  the 
cell  internal  form  representation  all  at  once,  immediately  giving  every  cell  a  monk  of  version  0.  But 
transformation  only  begins  at  one  subtree  of  the  parse  tree,  and  brothers  of  that  subtree  arc  not 
transformed  until  later.  To  maintain  the  continuity  of  the  range  indicated,  we  define  the  minimum 
of  a  cell's  range  to  be  only  the  time  of  the  first  actual  transformation  to  reach  that  cell  or  one  of  its 
subcclls. 

Two  slots  arc  left  open  in  a  cell  at  its  creation  for  the  eventual  storage  of  its  environment 
and  index  of  bindings.  Ihcse  will  be  installed  immediately  before  transformation  of  the  cell  is 
attempted.  The  function  SEX  PR  I RLCORD-ENVDEX  is  called  within  each  transformation  scL  and 
will  cause  the  two  structures  to  be  stored  in  the  input  cell.  Although  the  transformers  do  not 
access  this  common  contextual  information  via  the  cell  they  transform,  they  could  if  they  knew 
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how.  However,  it  seems  cleaner  to  keep  them  uninformed  about  the  temporary  duplication  of 
information.  I,atcr,  when  die  transformation  process  ends  and  a  user  wishes  to  obtain  common 
contextual  information  about  an  expression,  it  will  be  provided  him  from  the  copy  retained  in  that 
expression’s  cell. 


4.3.2.  The  Transformation  of  Cells 

How  is  it  possible  to  neatly  package  the  history  of  each  subtree  of  a  program  into  its  own 
cell,  given  that  any  number  of  unspecified  transformations  will  be  engaged  in  collapsing  code  or 
moving  pieces  of  the  program  from  place  to  place?  In  an  attempt  to  avoid  having  more  than  one 
cell  to  represent  a  particular  node  of  the  parse  tree,  yet  at  the  same  time  maintain  the  integrity  of  a 
node's  history,  I  have  developed  special  monks  which  keep  cells  clean  and  tidy.  These  monks  are 
created  as  needed  by  RESPONSE  1HACK;  the  transformers  need  not  concern  themselves  with  such 
low  level  details.  Indeed,  the  transformers  know  nothing  at  all  of  the  internal  representation  of  the 
argument  program. 

When  a  transformer  is  called  via  XFORM,  it  is  passed  the  cell  to  be  transformed,  and,  if  the 
transformer  performs  a  global  transformation,  the  environment  and  index  of  bindings  as  well.  The 
transformer's  response  consists  of  a  success/fail  flag,  the  resultant  expression,  and  a  justification.  If 
the  transformation  fails,  it  returns  the  same  cell  as  was  input  Otherwise  the  returned  object  must 
be  either: 

(1)  the  same  cell  given  to  the  transformer  (the  “input”  cell), 

(2)  a  cell  which  is  a  “subcell"  of  the  input  cell, 

(3)  a  newly  made  cell,  perhaps  copied  from  elsewhere  in  the  tree, 

(4)  a  list  of  cells,  or 

(5)  nothing. 

I  will  treat  each  of  these  cases  in  turn,  after  which  we  will  discuss  the  completion  of  histories. 


4.3.2. 1 .  Transformers  Which  Call  Slaves 

If  the  cell  returned  from  a  transformer  is  the  same  as  the  input  cell,  the  transformer  must  have 
called  slave  transformers  to  perform  a  local  transformation.  In  this  case,  a  new  monk  of  the  same 
type  as  before  will  be  created  and  installed  in  the  input  cell.  Though  the  subtree  pointed  to  by  the 
new  monk  is  die  same  one  die  last  monk  pointed  to.  the  new  monk  carries  with  it  the  new  version 
number,  transformation  name,  and  justification,  signifying  that  a  transformer  did  apply  to  the  cell. 

Any  transformation  performed  by  slave  transformers  will  have  been  recorded  in  the  cells 
which  were  input  to  those  slave  transformers,  diereby  updating  only  the  histories  of  die  relavent 
subcclls,  and  leaving  other  subcclls  of  die  parent  transformation  cell  both  unchanged  and 


(DEFUN  XFORM-SLAVE  (XFORMER  CELL  ARCS  PTC POINTER) 

(LET  ((RESPONSE  (APPLY  XFORMER  (CONS  CELL  ARGS)))) 

(COND  ((WAS-CHANGEO?  RESPONSE) 

(CELL! STORE -MONK 
CELL 

(RESPONSE! SLAVE -HACK  RESPONSE  CELL  XFORMER  PTCPOINTER)) 
(CLOAKRACKiHANG-SLAVE  CELL  XFORMTIME) 

( XFORMER ! RECORD  XFORMER  CELL  XFORMTIME)) 

(T  (CELL!STORE-H£RMIT 
CELL 

(RESPONSE ISLAVE-HACKF  RESPONSE  XFORMER  PTCPONTER) ) 
(RAGRACK! HANG -SLAVE  CELL  XFORMTIME) 

(  XFORMER ! RE CORDF  XFORMER  CELL  XFORMTIME))))) 

(DEFUN  SLAVE-RESPOND  (APPLIED?  OBJECT  JUST) 

(COND  ((NOT  APPLIED?)  (LIST  ••SLAVE-RESPONSE**  .FAILED*  JUST)) 

(T  (LIST  ..SLAVE-RESPONSE**  OBJECT  JUST)))) 

Table  4-10,  Slave  transformations  allow  the  histories  of  subcells  to  be  updated. 


uncopicd.  The  pointers  in  the  parse  tree  which  connect  the  subcells  to  the  parent  cell  remain 
intact.  The  use  of  slave  transformers  in  the  transformational  component  permits  the  accountable 
component  to  maintain  the  locality  of  transformation. 

Slave  transformers  arc  expected  to  return  succcss/fail  flags,  resultant  cells,  and  justifications, 
just  as  their  masters  do.  XFORM-SLAVE  installs  these  in  the  history  of  the  slave  transformer's  input 
cell,  along  with  a  pointer  to  the  parent  transformation  cell.  Kvcry  monk  has  such  a  "P  I'C'  pointer; 
A  non-nil  PTC  pointer  indicates  that  the  transformer  which  caused  the  monk's  creation  was  a  slave 
transformer.  In  particular,  it  was  a  slave  to  the  transformer  which  created  the  monk  of  the  same 
version  contained  in  the  parent  cell  pointed  to.  Thus,  for  example,  if  CELL-  14  contains  a  version 
5  monk  with  a  PTC  pointer  to  CELL-59,  we  know  that  CELL-59  contains  a  version  5  monk  as 
well,  and  the  transformer  indicated  in  that  monk  was  the  master  of  the  transformer  indicated  in 
CELL-  14  s  version  5  monk.  The  definition  of  XFORM-SLAVE  is  given  in  fable  4-10. 


4.3.2.2.  Transformers  Which  Simplify 

A  transformer  w  hich  returns  a  subccll  of  the  input  cell  was  able  to  simplify  the  expression  by 
“bypassing”  some  part  of  its  structure,  as  in  the  rule  (APPEND  <x>  NIL)  =>  <x>.  (1  use  the 
function  CELL !  FATHERS  to  determine  if  the  cell  returned  by  a  transformation  is  a  subccll  of  the 
input  cell.  If  the  input  cell  is  a  member  of  the  fathers  of  the  returned  cell,  then  the  returned  cell  is  a 
subccll.) 

How  can  this  returned  subccll  be  incorporated  into  the  input  cell?  If  we  install  the  tnost  recent 
monk  of  the  returned  subccll  as  the  new  monk  of  the  input  cell,  we  confuse  the  histories  of  the  tw  o 
pieces  of  code.  Someone  interested  only  in  following  the  progress  of  the  expression  <x>  would 
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suddenly  be  left  hanging  at  the  point  when  the  transformation  occurred,  unless  some  complicated 
method  of  backpointers  and  justifications  was  concocted.  My  decision  was  to  instead  create  a 
special  type  of  monk  called  a  ‘POINTER*.  The  cell  which  represents  <x>  is  left  unchanged, 
just  as  the  expression  itself  was  untouched  by  the  transformer.  The  new  monk  is  installed  in  the 
input  cell  along  with  the  normally  recorded  information.  The  subtree  of  this  •POINTER*  monk, 
however,  is  the  <x>  cell.  The  type  of  the  resultant  input  cell  is  defined  to  be  the  type  of  the  object 
of  the  *P0INTER*  monk,  and  someone  interested  in  following  die  history  ne  input  cell,  upon 
confronting  the  ‘POINTER*,  continues  his  study  by  picking  up  the  history  of  the  <x>  cell  at  the 
time  the  transformation  occurred.  The  history  of  the  <x>  cell,  however,  remains  totally  contained 
in  that  cell  (unless  it,  too,  is  simplified  to  point  to  a  subcell),  and  is  oblivious  to  the  effects  of 
transformations  performed  on  cells  above  it  in  the  parse  tree. 


4.3.2. 3.  Transformers  Which  Replace 

A  transformer  which  returns  a  new  cell,  perhaps  copied  from  an  expression  elsewhere  in  the 
parse  tree  or  newly  created,  intends  for  the  subtree  of  that  new  cell  to  completely  replace  the 
subtree  of  the  input  cell.  This  might  occur  in  situations  when  an  expression  is  being  replaced  by 
its  trudi  value,  when  a  procedure  definition  is  substituted  for  a  call,  or  when  actual  arguments  are 
being  substituted  for  formal  arguments,  for  example.  In  all  eases,  the  new  cell  should  be  "deeply” 
copied  (that  is,  copied  at  all  levels)  from  its  source  so  diat  future  transformations  which  apply  to 
it  or  its  subcclls  in  the  new  context  will  not  affect  the  original  copy.  Transformers  which  perform 
such  replacements  or  substitutions  are  responsible  for  providing  the  new  copy  of  the  returned  cell 
(note  die  calls  to  die  function  SEXPR1C0PY  in  Table  4-1).  Copied  cells  in  some  sense  begin 
their  existence  at  the  time  of  their  substitution,  and  thus  are  not  responsible  for  the  history  of  the 
original  cell.  Thus,  the  latest  monk  of  die  copied  cell  is  installed  as  the  new  monk  in  die  input  cell, 
and  the  copied  cell  itself  is  thrown  away.  Hie  justification  for  die  transformation  should  provide  the 
information  necessary  to  determine  the  source  of  the  new  cell  should  the  user  desire  it. 


4.3.2.4.  Transformers  Which  Lislify 

The  object  returned  from  a  transformer  may  be  a  list  of  cells,  indicating  that  one  cell  is  to  be 
replaced  by  many.  Under  what  circumstances  might  this  occur?  A  transformer  which  returns  a  list 
of  cells  is  nearly  always  a  slave  transformer,  but  since  slaves  are  treated  almost  exactly  as  regular 
transformers,  it  will  not  enter  into  die  discussion  here.  Recall  the  example  given  in  secuons  3.4.2 
and  3.4.4  where  this  occurred  with  conditional  clauses: 

(COND  <clause$-l>  (T  (CONO  <clauses-2>))) 

=>  (COND  <clauses-l>  <clauses-2>) 


* 
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A  transformer  to  do  this  would  be  placed  in  the  *COND*  transformation  set,  and  after  verifying 
that  the  input  expression  is  of  the  correct  form,  calls  a  slave  transformer  on  the  final  clause  K  of 
the  input  conditional  expression.  The  slave  transformer  returns  the  list  of  clauses  <clauses-2>. 
Clause  K  should  have  a  special  monk  installed  in  its  history  to  show  that  n  now  points  to  the  list 
<clauses-2>.  To  allow  a  cell  to  point  to  a  list  of  subtrees.  1  have  created  another  special  monk 
called  a  .POINTER-LIST., 

Another  case  in  which  a  slave  transformer  is  used  to  return  a  list  of  expressions  is  in  im¬ 
plementation  of  the  transformation  rule: 

(PLUS  (PLUS  <Y>  <Z>)  <X>)  =>  (PLUS  <X>  <Y>  <Z>) 

A  transformer  to  perform  this  transformation  would  appear  in  the  »FCALL*  transformation  set 
and  apply  to  the  outer  function  call,  but  would  call  a  slave  transformer  on  the  inner  function  call 
that  returns  a  list  of  its  arguments. 

The  use  of  .POINTER-LIST*  monks  complicates  accessing  and  conversion  functions  which 
now  must  be  able  to  suddenly  handle  a  list  of  subtrees  w  hen  they  were  only  expecting  a  single  tree. 
A  special  MAPCAR  function  was  written  w-hich  checks  for  .POINTER-LIST*  cells  before  passing 
them  as  arguments  to  the  stated  function.  Because  such  system  functions  must  be  able  to  detect 
and  deal  with  .POINTER-LIST,  cells,  the  type  of  such  cells  is  defined  to  be  .POINTER-LIST., 
rather  than  the  type  of  the  first  subtree  in  the  list  of  subtrees  pointed  to.  A  user  of  the  accountable 
component,  however,  need  not  know  of  their  existence,  and  will  receive  expected  type  information 
in  answer  to  the  same  query. 


4.3. 2.5.  Transformers  Which  Delete 

A  transformer  which  returns  nothing  must  do  so  by  returning  a  “nothing"  flag,  which  occurs 
in  this  system  as  the  special  variable  ..NOTHING**.  Such  transformers  currently  must  be  slaves, 
but  again,  that  will  not  enter  into  our  discussion. 

When  a  nothing  flag  is  returned,  a  special  .NOTHING*  monk  is  created  and  installed  in  the 
input  cell,  complete  with  version  number,  transformer  name,  etc.  just  as  though  nothing  unusual 
had  happened,  flic  subtree  pointed  to  by  a  .NOTHING*  monk  is  NIL.  and  the  type  of  cells  which 
contain  .NOTHING,  monks  is  defined  to  be  .NOTHING*.  No  transformer  can  ever  apply  to  such 
a  cell  again  since  the  cell  will  never  be  detected;  it  no  longer  exists  as  far  as  the  transformational 
component  is  concerned. 

Hie  use  of  .NOTHING*  monks  is  currently  restricted  to  tire  case  in  which  the  deleted  expres¬ 
sion  occurred  as  an  clement  of  a  list  in  die  program.  That  is,  as  an  clement  of  an  argument  list 
to  a  function  call,  a  dummy  argument  list,  a  list  of  conditional  clauses,  etc.  This  allows  system 
functions  which  manipulate  cells  to  correctly  ignore  the  nothing  cel),  which  is  actually  still  very 
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much  a  something.  This  restriction  should  not  hamper  the  transformation  implementor,  since  any 
transformation  which  deletes  an  object  not  in  a  list  can  be  implemented  as  a  transformer  which 
simplifies  (the  expressions  such  transformers  bypass  are  essentially  deleted). 


4.3. 2. 6.  Completing  Histories 

We  have  discussed  the  maintenance  of  cell  histories  by  installing  monks  with  information 
regarding  the  success  or  failure  of  transformations  to  those  cells.  However,  there  is  another  pos¬ 
sibility  which  1  have  not  yet  provided  for.  It  may  be  the  ease  that  a  CELL- 17  is  eliminated 
from  the  parse  tree  by  a  transformation.  When  the  resulting  parse  tree  is  converted  back  to  ex¬ 
ecutable  LISP  code,  CELL-17  will  properly  be  omitted,  since  it  is  not  pointed  to  by  any  of  the 
current  monks  of  cells  above  it.  However,  someone  interested  in  following  only  the  history  of 
CELL- 17  will  not  be  aware  of  the  point  at  which  some  cell  higher  in  the  parse  tree  ceases  to 
reference  CELL -17.  Reaching  the  last  monk  in  the  cell,  that  person  will  assume  (and  rightly  so) 
that  CELL- 17  occurs  in  the  final  parse  tree  as  the  subtree  recorded  by  that  monk. 

For  example,  if  the  *AND*  cell  which  represents  the  expression  (AND  T  (CONS  X  Y)) 
is  replaced  by  T  (assuming  that  the  expression  is  in  predicate  position),  then  the  *ATOM*  T, 
the  *FCALL*  expression  (CONS  X  Y),  die  .FUNCNAME*  CONS,  and  die  *ATOM*s  X  and  Y 
should  all  have  dicir  histories  brought  to  an  end.  To  uo  diis,  I  have  created  a  special  monk  of  type 

♦  REPLACED*.  Hie  subtree  of  this  monk  is  empty,  and  when  installed  in  a  cell  simply  signifies  that 
some  transformation  occurred  which  caused  this  cell  to  be  replaced.  W'hcncvcr  a  transformation 
returns  a  new  cell  or  a  nothing  flag,  the  histories  of  all  the  subcells  of  the  input  cell  are  completed 
by  installing  tlic  *REPLACED*  monk. 

Now  consider  the  transformation  (APPEND  NIL  <x>)  =>  <x>.  This  transformation 
bypasses  cells  to  return  a  subccll  of  the  input  cell,  as  in  section  4.3.2.2.  In  order  to  bring  the 
histories  of  the  *FUNCNAME*  cell  containing  APPEND  and  die  *ATOM*  cell  containing  NIL  to 
a  close.  1  have  created  another  special  monk  called  a  *BYPASS*  monk.  Again,  the  *BYPASS* 
monk  has  an  empty  subtree,  and  when  installed  in  a  cell  signifies  that  diis  cell  was  bypassed  by  a 
simplify  ing  transformer  and  therefore  docs  not  remain  in  die  resultant  program. 

In  general,  the  history  of  any  cell  which  becomes  invisible  to  future  transformations  should 
be  "completed"  with  one  of  the  special  monks.  The  transformers  themselves  are  not  respon¬ 
sible  for  this;  they  need  know  nothing  of  the  existence  of  cells  at  all.  since  die  transformational 
component  should  be  oblivious  to  die  existence  of  die  accountable  component.  Instead,  all 
such  housekeeping  (cell-keeping!)  is  performed  by  the  function  CELL !  TERMINATE,  which  is 
called  by  RESPONSE  ! HACK  and  RESPONSE  !  SLAVE -HACK.  These  latter  two  arc  able  to  deter¬ 
mine  which  types  of  the  possible  special  monks  *P0INTER*,  *REPLACED*.  *BYPASSED*, 

♦  POINTER-LIST*,  or  *N0THING*  to  create,  simply  by  determining  in  which  of  the  five 
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(OEFUN  RESPONSE ! HACK  (RESPONSE  INPUTCELL  XFORMER  PTCPOINTER) 

(LET 

((OBJECT  ( RESPONSE (OBJECT  RESPONSE)) 

(JUST  (RESPONSE! JUST  RESPONSE))) 

(CONO 

((EQ  OBJECT  INPUTCELL)  ;  must  have  been  a  master  transformer 
(COND  ( (RESPONSE ! SLAVE ?  RESPONSE) 

(ERROR  '(Slaves  can't  return  same  cell  --  RESPONSE  .'HACK  | )) ) 
(HONKtCREATE 

(SEXPR1TYPE  OBJECT) 

XFORMTIME 

(SEXPRITREE  OBJECT) 

XFORMER 

JUST 


••MASTER** ) ) 

( ( CELL ! SUBTREE?  INPUTCELL  OBJECT)  ;  Sign  of  a  ••POINTER.* 

( CE  LL ! STORE -BP  OBJECT  ( CELL  !  BP  INPUTCELL)) 

(CELL ! TERMINATE  INPUTCELL  (LIST  OBJECT)  PTCPOINTER) 

(MONKiCREATE  ..POINTER..  XEORMTIME  OBJECT  XFORMER  JUST  PTCPOINTER)) 

((CELL?  OBJECT)  ;  must  be  a  new  cell 
(CELL ! TERMINATE  INPUTCELL  NIL  PTCPOINTER) 

(MONKICREATE  (SEXPRITYPE  OBJECT) 

XFORMTIME 

(SEXPRITREE  OBJECT) 

XFORMER 

JUST 

PTCPOINTER)) 

((CELLIST?  OBJECT)  ;  sign  of  a  .POINTER-LIST* 

(MAPC  (FUNCTION  (LAMBDA  { POINTEE ) 

(CELL ISTORE-BP  POINTEE 

(CELL  IBP  INPUTCELL)))) 


OBJECT) 

(CELL  I  TERMINATE  INPUTCELL  OBJECT  PTCPOINTER) 

(MONKICREATE  ..POINTER-LIST..  XFORMTIME  OBJECT  XFORMER  JUST  PFCPOIHTER)) 
(T  (ERROR  'IUnknown  response  object  —  RESPONSE  I  HACK  |  OBJECT))))) 


(DEFUN  RESPONSE ISLAVE-HACK  (RESPONSE  INPUTCELL  XFORMER  PTCPOINTER) 

(LET  ((OBJECT  (RESPONSE IOBJECT  RESPONSE)) 

(JUST  (RESPONSE  I  JUST  RESPONSE))) 

(COND  ((EQ  OBJECT  ..BYPASS**) 

(MONKICREATE  ..BYPASS*.  XFORMTIME  NIL  XFORMER  JUST  PTCPOINTER)) 
((OR  (EQ  OBJECT  ..NOTHING*. )  (EQ  OBJECT  ..REPLACED.. ) ) 

(CELL I TERMINAT E  INPUTCELL  NIL  PTCPOINTER) 

(MONKICREATE  OBJECT  XFORMTIME  NIL  XFORMER  JUST  PTCPOINTER)) 

(T  (RESPONSE  I  HACK  RESPONSE  INPUTCELL  XFORMER  PTCPOINTER))))) 


Tablp.4-11.  Functions  which  perform  “cell-keeping”. 


categories  the  returned  object  falls. 

The  definitions  of  the  functions  RESPONSE  1  HACK  and  RESPONSE  ISLAVE-HACK  arc given 
in  Table  4-11.  After  determining  the  type  of  response  object  returned  by  the  transformer  (same 
cell,  subccll,  new  cell,  list  of  cells,  etc.),  up  to  three  things  happen: 
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(DEFUN  CELL ! TERMINATE  (CELL  •  tERM-OBJL 1ST.  • T E RM- PTCP01 NT ER. ) 

(DECLARE  (SPECIAL  .TERM-0BJL1ST.  .TERM- PTCPOINT ER. ) ) 

(COND 

((NULL  .TERM-OBJLIST.) 

( SEXPR !MAPCAR 
(FUNCTION 
(LAMBDA  (SON) 

(XFORM-SLAVE  < SEXPR ! REPLACED 
SON 
(LIST) 

•TERM -PTC POINTER.))) 

(CELL ! CSONS  CELL))) 

(T  (SEXPRIMAPCAR 
(FUNCTION 
(LAMBDA  (SON) 

(COND  ((MEMO  SON  .TERM-OBJLIST.)  NIL) 

(T  (XFORM-SLAVE  ' SEXPR ! BYPASS 
SON 
(LIST) 

•TERM-PTCPO INTER.) 

(CELL ! TERMINATE  SON 

•TERM-OBJLIST. 

•  TERM-PTCPOINTER. ))))) 

(CELL ! CSONS  CELL))))) 

Table 4-12.  Cell  histories  are  compleied  using  CELL  !  TERMINATE. 


•  back-pointers  are  updated.  The  back-pointer  list  of  the  returned  object  is  updated  to  include 
references  to  the  input  cell,  since  the  object’s  subtree  is  about  to  be  installed  in  the  input  cell  via 
a  new  monk. 

•  the  rclavent  old  subtrees  of  the  input  cell  arc  terminated.  CELL ! TERMINATE  takes  a  cell 
whose  current  subcells  arc  to  be  terminated,  a  list  of  subcclls  of  the  input  cell  which  are  not 
to  be  terminated  (it  checks  each  of  the  former  against  the  latter  before  terminating),  and  the 
input  cell  which  acts  as  the  parent  transformation  cell  for  llic  termination  process.  The  special 
termination  monks  .BYPASS*  and  .REPLACE*  arc  installed  via  system  defined  slave  trans¬ 
formers  which  use  the  FI’C  pointer  information.  The  definition  of  CELL  !  TERMINATE  is  given 
in  fable  4-12. 

•  the  new  monk  is  created  and  returned  to  XFORM  (or  XFORM-SLAVE),  which  will  then  install  it 
in  the  input  cell. 


4.4.  Retrieval  of  Information 


At  the  start  of  the  research  for  this  thesis.  1  had  grand  ideas  of  a  system  in  which  the  user 
had  only  to  hint  at  what  he  didn’t  understand  about  the  transformation  process.  The  accountable 
component  would  immediately  understand  the  source  of  his  confusion  and  provide  him  with  the 
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information  he  needed.  This,  1  thought,  would  be  a  reasonable  goal;  very  Al-ish,  and  much 
easier  than  the  even  more  idealistic  goal  in  which  the  system  could  detect  "unusual”  transformation 
situations  and  explain  them  without  any  prompting  by  the  user. 

Such  goals  may  yet  he  attained,  however,  for  the  present  I  have  accepted  tire  fact  that  provid¬ 
ing  simple  functions  for  accessing  the  information  stored  in  cells  and  monks  will  have  to  be 
sufficient.  Given  a  cell  and  a  transformation  time,  for  example,  the  function  DISP-JUST  will 
display  the  justification  for  the  transformation  which  occurred  to  dial  cell  at  that  time.  Or.  given  a 
transformation  time,  the  function  LOCATE-NEWCELL  will  return  the  cell  which  was  transformed 
at  that  time.  There  arc  a  many  such  functions  for  retrieving  information;  their  usefulness,  however, 
is  limited  to  those  people  who  know  which  ones  to  call  to  get  the  desired  information.  The  user 
should  be  able  to  request  information  with  the  least  amount  of  hassle.  Obviously  then,  the  retrieval 
functions  should  be  packaged  up  and  presented  via  a  menu  of  some  sort. 


4.4.1.  The  Menus 

Following  the  transformation  of  a  list  of  evaluable  s-expressions  (the  input  accepted  by  the 
driver  function  TRANSFORM),  the  resulting  list  of  cells  (one  for  each  s-expression)  is  bound  to  the 
atom  XSEXPR.  The  transformed  cells  arc  cross-referenced  by  transformation  time  in  CLOAKRACK, 
and  cells  to  which  transformers  applied  but  failed  are  cross-referenced  by  transformation  time  in 
RAGRACK.  In  addition,  a  list  of  the  cells  and  times  for  which  each  transformer  succeeded  or  failed 
to  apply  has  been  recorded  in  the  properly  list  associated  with  the  name  of  that  transformer.  To 
give  the  user  access  to  the  information  stored  in  these  structures,  the  query  element  presents  the 
user  with  a  number  of  choices  via  the  Menu  Menu. 

The  Menu  Menu  allows  the  user  to  select  either  the  Code  Menu,  the  Transformer  Menu, 
or  the  Version  Menu.  Kach  of  these  present  the  user  w  ith  his  choice  of  code,  transformer,  or 
transformation  time,  respectively,  to  be  die  subject  of  his  query.  If  he  selects  the  Code  Menu, 
he  may  then  specify  any  one  of  the  expressions  from  the  list  of  those  he  gave  to  the  function 
TRANSFORM.  Thus,  for  example,  if  he  transformed  a  list  of  five  expressions,  he  may  enter  “3"  to 
point  to  the  third  expression.  The  Transformer  Menu  gives  the  user  access  to  statistics  concerning 
the  number  of  transformers  given  to  the  system,  the  number  that  were  applied,  and  how  many 
always  failed  or  always  applied.  The  user  may  ask  to  sec  a  list  of  the  names  of  the  transformers  that 
fall  into  any  of  these  categories,  or  enter  one  transformer  name  and  study  its  activities.  The  Version 
Menu  will  tell  the  user  what  the  final  transformation  time  was.  then  request  a  version  number  as 
input.  The  expression  transformed  at  that  time  will  be  displayed,  and  the  user  may  continue  to  ask 
questions  about  that  expression,  or  return  to  the  menu  to  inquire  about  a  new  transformation  time. 
A  diagram  of  the  menus  is  given  in  Figure  4-2. 

The  functions  used  to  support  the  menu  system  arc  simple  and  straightforward.  1  defined  a 


r  —%^-rir 


general  menu  display  function,  D ISP -MENU,  which  takes  a  title,  a  list  of  selections,  and  a  list  of 
messages  to  he  displayed  after  the  selections.  This  function  is  used  by  each  of  the  menu  functions 
MFNU-ME.NU,  CODE-MENU,  TRANSFORMER-MENU,  and  VERSION-MENU.  If  the  user  selects  an 
expression  from  the  Code  Menu,  that  expression  will  he  passed  to  the  function  WALK.  1  his  func¬ 
tion  will  allow  the  user  to  obtain  a  variety  of  information  stored  in  the  cell  which  rcptercnls  that 
expression.  If  the  user  selects  a  transformer  from  the  Transformer  Menu,  he  will  be  given  a  number 
of  statistics  on  the  activities  of  that  transformer,  and  then  will  he  allowed  to  “walk"  any  of  the 
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expressions  transformed  by  that  transformer.  And  finally,  if  the  user  selects  a  transformation  time 
from  the  Version  Menu,  he  will  walk  the  cell  which  was  transformed  at  that  time. 


4.4.2.  Walking 

The  function  WALK  takes  any  cell  and  optionally,  a  version.  If  no  version  is  given,  he  will 
begin  walking  at  version  0.  The  expression  will  be  displayed  along  with  the  version  number, 
the  expression's  structure  type,  and  the  name  of  die  transformer  which  created  it.  Hie  user  may 
then  enter  any  of  the  commands  described  in  Chapter  Tw'o  (and  some  others  besides)  to  obtain 
additional  information. 

The  simplest  method  of  retrieving  information  from  the  cells  is  for  the  user  simply  to  enter 
“  +  ”  at  each  prompt,  which  w  ill  display  the  expression  as  it  occurred  in  the  next  version.  This  has 
the  effect  of  simulating  the  transformation  process  as  it  applied  to  the  expression.  If,  as  the  user 
walks  through  this  process,  step  by  step,  he  wishes  to  stop  along  the  way  and  inquire  in  more  detail 
about  some  transformation .  he  may  easily  do  so.  Kntcring  “w  "  (for  “why”),  for  example,  will  cause 
the  justification  for  the  current  transformation  to  be  displayed.  Kntcring  “f  (for  "failed”)  will 
cause  the  names  of  the  transformers  which  were  attempted  but  failed  at  this  point  in  the  process 
to  be  displayed  along  with  their  justifications.  F.ntering  “?”  will  display  the  menu  of  commands 
available  to  the  user. 

If  the  user  has  a  pretty  good  idea  of  what  he  wants  to  look  at,  he  may  go  directly  to  piece  of 
code  he  is  interested  in  by  using  the  "up”,  "down",  “over",  and  "set  version"  commands  described 
in  Chapter  Two.  These  allow  him  to  pick  up  his  walk  anywhere  in  time  (transformation  time)  or 
space  (point  in  code).  If  at  any  time  during  his  walk  he  w  ishes  to  jump  down  to  the  sub-expression 
which  was  just  transformed,  the  user  may  enter  the  command  "j"  (for  "jump"),  rather  than  the 
correct  sequence  of  downs  and  overs.  After  exploring  around  all  he  wants,  he  may  enter  “r”  (for 
return)  to  return  to  the  original  expression.  And.  as  a  special  added  feature,  the  mechanism  used 
to  remember  the  expression  jumped  from  (a  stack!)  was  generalised  and  made  available  to  the  user 
via  the  command  “m"  (for  memorize).  This  adds  the  current  expression  and  version  number  to  the 
stack  (stored  in  CELL-ENVIRONMENT)  without  jumping  anywhere.  The  user  may  walk  anywhere 
he  likes,  and  then  enter  “r”  to  return  the  last  expression  and  version  which  were  either  jumped 
from  or  "memorized”. 


4  5.  An  Example 

"■  !•  i  tuiKiiun  of  the  system  is  TRANSFORM.  1  call  that  here  on  the  list: 
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(COND  {{NULL  Y)  y) 

(T  (F00  (CAT  Y))))) 

(DEFUN  FOO  (X) 

(CONS  XX)) 

(OEFUN  BAR  (FR08) 

(COND  ((ATOM  FROB)  (TEST  FROB  FROB)) 

((OR  FROB  (COR  FROB))  (CAR  FROB)) 

(T  (CAR  (FOO  FROB)))))) 

which  is  bound  to  the  atom  OAFS. 

(transform  dafs) 

Transformation  completed. 

When  the  transformation  process  is  complete,  the  user  may  enter  the  menu  system,  invoking  the 
query  element. 

Enter  Menu  System?  (enter  y  or  n)  j 
THE  MENU  MENU 

0  -  Quit 

1  -  Code  Menu 

2  -  Transformer  Menu 

3  -  Version  Menu 
Selection?  (end  with  CR):  1 

Upon  entering  the  code  menu,  die  name  of  each  function  definition  transformed  (or  an  expression 
type  if  the  the  expression  transformed  is  not  a  function  definition)  will  be  displayed  so  the  user  may 
select  one  of  them  to  examine. 

CODE  MENU 

0  -  Return  to  Menu  Menu 

1  -  TEST 

2  -  FOO 

3  -  BAR 

Enter  Q  to  quit. 

Selection?  (end  with  CR):  3 

The  selected  expression  is  first  displayed  as  it  was  input  to  the  transformation  system. 

(DEFUN  BAR  (FROB) 

(COND  ((ATOM  FROB)  (TEST  FROB  FROB)) 

((OR  FROB  (COR  FROB))  (CAR  FROB)) 

(T  (CAR  (FOO  FROB))))) 
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Version:  0  Expression  type:  *DEFETTE«  Transformer:  DEFETTE 1CREATE 
:? 

'Hie  user  may  type  “?"  at  any  time  to  see  a  list  of  possible  commands. 

d  -  down  to  first  sub-expression  of  this  cell 

u  -  up  to  parent  expression 

o  -  over  to  brother  expression 

j  -  jump  jump  to  sub-expression  just  changed, 

r  -  return  return  to  expression  last  jumped  from  or  memorized 
+  -  add  increment  version  by  one  and  print 

-  -  subtract  decrement  version  by  one  and  print 
s  -  set  set  version  to  number  prompted  for 

n  -  next  go  to  next  version  of  this  cell  and  print 
p  -  previous  go  to  previous  version  of  this  cell  and  print 
1  -  last  go  to  last  version  of  this  cell  and  print 
m  -  memorize  remember  this  cell  and  this  version 

a  -  again  print  current  version  of  expression 

h  -  how  many  display  version  numbers  for  this  expression 
t  -  type?  what  is  type  of  top  level  expression 

v  -  version?  what  is  version  of  top  level  exprssion? 

w  -  why?  get  justification  of  current  transformation 
x  -  xformer?  what  transformer  produced  this  cell's  current  monk, 
e  -  evaluate  display  the  boolean  value  of  this  expression 
f  -  fails  look  at  the  transformers  which  failed 

b  -  break  break  to  LISP 

q  -  quit  quit  to  top  level 

?  -  huh?  prints  this  info 

:  + 

The  user  may  rc-cnact  the  transformation  process  by  entering  a  “  +  "  at  each  prompt 


(DEFUN  BAR  (FROB) 

(CONO  ((ATOM  FROB) 

((LAMBDA  (X-Z  Y-l) 

(COND  ((NULL  Y-l)  Y-l) 

(T  (FOO  (CAT  Y-l)))))  FROB  FROB)) 
((OR  FROB  (CDR  FROB))  (CAR  FROB)) 

(T  (CAR  (FOO  FROB))))) 


Version:  3  Expression  type:  »DEFETTE»  Transformer:  FUNCNAME 1SUB-DEF 

:  + 
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(DEFUN  BAR  (FROB) 

(COND  ((ATOM  FROB) 

((LAMBDA  (X-2  Y'l) 

(COND  ((NULL  Y-l )  NIL) 

(T  (FOO  (CAT  Y-l)))))  FROB  FROB)) 

((OR  FROB  (CDR  FROB))  (CAR  FR08)) 

(T  (CAR  (FOO  FROB))))) 

Version:  4  Expression  type:  *ATOM»  Transformer:  SEXPR! FORM-BVAL 


(DEFUN  BAR  (FROB) 

(CONO  ((ATOM  FROB) 

((LAMBDA  (X-2  Y-l) 

(COND  ((NULL  Y-l)  NIL) 

(T  ((LAMBDA  (X-3)  (CONS  X-3  X-3 ) )  (CAT  Y-l))))) 

FROB 

FROB)) 

((OR  FROB  (CDR  FROB))  (CAR  FROB)) 

(T  (CAR  (FOO  FROB))))) 

Version:  5  Expression  type;  *DEFETTE*  Transformer:  FUNCNAME I SUB-DEF 


(DEFUN  BAR  (FROB) 

(COND  ({ATOM  FROB) 

((LAMBDA  NIL 

(COND  ((NULL  FROB)  NIL) 

(T  ((LAMBDA  (X-3)  (CONS  X-3  X-3))  (CAT  FROB))))))) 
((OR  FROB  (CDR  FROB))  (CAR  FROB)) 

(T  (CAR  (FOO  FROB))))) 

Version:  6  Expression  type:  »FCALL»  Transformer:  DEFCALLITRIM 


(DEFUN  BAR  (FROB) 

(COND  ((ATOM  FROB) 

(COND  ((NULL  FROB)  NIL) 

(T  ((LAMBDA  (X-3)  (CONS  X-3  X-3))  (CAT  FROB))))) 

((OR  FROB  (CDR  FROB))  (CAR  FROB)) 

(T  (CAR  (FOO  FROB))))) 

Version:  7  Expression  type:  *COND»  Transformer:  DEFCALLISIMPLIFY 
:  + 
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(DEFUN  BAR  (FROB) 

(COND  ((ATOM  FROB) 

(CONO  ((NULL  FROB)  NIL) 

(T  ((LAMBDA  (X-3)  (CONS  X-3  X-3))  (CAT  FROB))))) 

((OR  T  (CDR  FROB))  (CAR  FROB)) 

(T  (CAR  (FOO  FROB))))) 

Version:  B  Expression  type:  *ATOM*  Transformer:  SEXPR 1 FORM-BVAL 


(DEFUN  BAR  (FROB) 

(CONO  ((ATOM  FROB) 

(CONO  ((NULL  FROB)  NIL) 

(T  ((LAMBDA  (X-3)  (CONS  X-3  X-3))  (CAT  FROB))))) 
((OR  T)  (CAR  FROB)) 

(T  (CAR  (FOO  FROB))))) 

Version:  9  Expression  type:  «OR*  Transformer:  ORITRIM 
:  + 


(DEFUN  BAR  (FROB) 

(COND  ((ATOM  FROB) 

(COND  ( ('NULL  FROB)  NIL) 

(T  ((LAMBDA  (X-3)  (CONS  X-3  X-3))  (CAT  FROB))))) 

(T  (CAR  FROB)) 

(T  (CAR  (FOO  FROB))))) 

Version:  10  Expression  type:  *ATOM»  Transformer:  ORISIMPLIFY 
:  + 


(DEFUN  BAR  (FROB) 

(COND  ((ATOM  FROB) 

(COND  ((NULL  FROB)  NIL) 

(T  ((LAMBDA  (X-3)  (CONS  X-3  X-3))  (CAT  FROB))))) 
(T  (CAR  FROB)))) 

Version:  11  Expression  type:  »COND»  Transformer:  C0ND1TRIM 
:  + 


No  more  transformations  occurred.  This  is  the  final  version. 

(DEFUN  BAR  (FROB) 

(COND  ((ATOM  FROB) 
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(CONO  ((NULL  FR08)  NIL) 

(T  ((LAMBDA  (X-3)  (CONS  X-3  X-3))  (CAT  FROB))))) 
(T  (CAR  FROB}):) 

Version:  11  Expression  type:  »C0ND»  Transformer:  CONDSTRIM 


If  the  user  requests  a  non-existent  version  of  the  expression,  he  is  warned  as  shown  above,  and  the 
expression  is  redisplayed.  When  he  is  through  walking  throught  the  transformation  process,  the 
user  quits  by  entering  “q"  and  is  returned  to  the  code  menu. 

CODE  MENU 

0  '  Return  to  Menu  Menu 

1  -  TEST 

2  -  F00 

3  '  BAR 

Enter  Q  to  quit. 

Selection?  (end  with  CR):  0 


THE  MENU  MENU 

0  -  Quit 

1  -  Code  Menu 

2  -  Transformer  Menu 

3  -  Version  Menu 
Selection?  (end  with  CR):  3 


VERSION  MENU 

There  are  11  versions. 

0  -  Return  to  Menu  Menu 
Q  -  quit 

or,  enter  version  number.. 

Selection?  (end  with  CR):  7 

The  version  menu  allows  the  user  to  begin  walking  the  code  at  a  particular  point  in  time  and  space: 
the  expression  transformed  at  time  7,  in  this  case.  After  the  code  is  displayed,  the  user  may  enter 
any  of  the  usual  walking  commands.  For  example,  he  might  enter  "u"  to  see  more  of  the  context  of 
the  expression  which  was  transformed. 


(COND  ((NULL  FROB)  NIL) 

(T  ((LAMBDA  (X-3)  (CONS  X-3  X-3))  (CAT  FROB)))) 
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Version:  7  Expression  type:  »C0ND*  Transformer:  DEFCALL I  SIMPLIFY 


((ATOM  FROB) 

(COND  ((NULL  FROB)  NIL) 

(T  {(LAMBDA  (X-3)  (CONS  X-3  X-3))  (CAT  FROB))))) 

Version:  7  Expression  type:  »C0ND»  Transformer:  DEFCALL I  SIMPLIFY 

:u 


(COND  ((ATOM  FROB) 

(COND  ((NULL  FROB)  NIL) 

(T  ((LAMBDA  (X-3)  (CONS  X-3  X-3))  (CAT  FROB))))) 

((OR  FROB  (CDR  FROB))  (CAR  FROB)) 

(T  (CAR  (FOO  FROB)))) 

Version:  7  Expression  type:  »COND»  Transformer:  DEFCALL ! SIMPLIFY 


To  sec  what  the  code  looked  like  immediatly  before  the  transformation,  the  user  enters 

(COND  ((ATOM  FROB) 

((LAMBDA  NIL 

(COND 

{(NULL  FROB)  NIL) 

(T  ((LAMBDA  (X-3)  (CONS  X-3  X-3))  (CAT  FROB))))))) 
((OR  FROB  (CDR  FROB))  (CAR  FROB)) 

(T  (CAR  (FOO  FROB)))) 

Version:  6  Expression  type:  »FCALL»  Transformer:  DEFCALUTRIM 


The  user  may  explore  as  much  as  he  desires,  and  when  he  quits  he  will  be  returned  to  the  menu  he 
came  from. 


VERSION  MENU 

There  are  11  versions. 

0  -  Return  to  Menu  Menu 
0  -  quit 

or,  enter  version  number. 
Selection?  (end  with  CR):  0 


86 


Chapter  Four 


The  Accouni  oile  Component 


THE  MENU  MENU 

0  -  Quit 

1  -  Code  Menu 

2  -  Transformer  Menu 

3  -  Version  Menu 
Selection?  (end  with  CR):  2 


TRANSFORMER  MENU 
0  -  Return  to  Menu  Menu 

1  -  Transformers  given  in  transformation  sets 

2  -  Transformers  attempted 

3  -  Transformers  which  always  failed 

4  -  Transformers  which  always  succeeded 

Enter  transformer  name  to  view. 

Selection?  (end  with  CR):  2 


1110  transformer  menu  gives  the  user  access  to  a  variety  of  statistics.  Rach  of  the  categories  listed 
above  will  display  the  number  of  transformers  in  that  category  and  give  their  names. 


14  transformers  attempted: 

( OR ! S IMPL I F Y  OR ! TRIM 

St  XPR ! REPLACED 
SEXPRIBYPASS 
ATOM! SUB-ACTARG 
St  X  PR ! NOTHING 
CONDI  SIMPLIFY 
CONDI  TRIM 
DEFCAUISIMPIIFY 
DE  FCALL I  TRIM 
FC ALL ! SIMPL IFY 
FCALL I S IMP-NOT 
FUNCNAME I SUB-DEF 
SEXPRIFORM-BVAL) 

Enter  any  character  to  continue:  x 


TRANSFORMER  MENU 


0  -  Return  to  Menu  Menj 

1  -  Transformers  given  in  transformation  sets 

2  -  Transformers  attempted 

3  -  Transformers  which  failed 

4  -  Transformers  which  succeeded 
Enter  Q  to  quit. 

Enter  transformer  name  to  view. 

Selection?  (end  with  CR):  4 


11  transformers  succeeded: 

(OR ! SIMPLIFY  OR ! TRIM 

SEXPRIREPLACED 
SEXPR18YPASS 
ATOM  1  SUB -ACT AR6 
SEXPR 1  NOTHING 
CONDI  TRIM 
DEFCALL I  SIMPLIFY 
DEFCALL I  TRIM 
FUNCNAME I SUB-DEF 
SEXPRIFORM-BVAL) 


6  transformers  always  succeeded: 

(OR  1 SIMPL IF Y  OR  1  TRIM 

SEXPRIREPLACED 

SEXPRIBYPASS 

ATOMISUB-ACTARG 

SEXPRINOTHING) 


Enter  any  character  to  continue:  x 


TRANSFORMER  MENU 
0  -  Return  to  Menu  Menu 

1  -  Transformers  given  in  transformation  sets 

2  -  Transformers  attempted 

3  -  Transformers  which  failed 

4  *  Transformers  which  succeeded 
Enter  0  to  quit. 

Enter  transformer  name  to  view. 

Selection?  (end  with  CR):  or!lri»n 
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If  the  user  enters  the  name  of  a  transformer,  he  is  taken  to  the  transformer  activities  menu  and 
given  some  statistics  on  the  performance  of  that  transformer. 

TRANSFORMER  ACTIVITIES  MENU 

0  -  Return  to  Transformer  Menu 

1  -  Observe  all  1  attempts  of  transformer  ORITRIM 

2  -  Only  its  1  successful  applications 

3  -  Only  its  0  unsuccessful  applications 
Enter  Q  to  quit. 

Selection?  (end  with  CR):  2 


The  user  may  study  both  successful  and  unsuccessful  applications  of  the  transformer. 

The  transformer  ORITRIM  applied  at  time  9. 

Would  you  like  to  study  the  expression?  (enter  y  or  n)  y 

(OR  T) 

Version:  II  Expression  type:  »0R»  Transformer:  ORITRIM 

(OR  T  (COR  FROB)) 

Version:  10  Expression  type:  *AT0M»  Transformer:  SEXPR I FORM-BVAL 

:w 


While  walking,  entering  the  command  “w"  (for  "why?")  will  produce  a  set  of  justifications  for  the 
transformation. 


Expression  was  transformed  because  (not  (atom  x))  ■>  (not  (null  x ) )  *>  x 
(ATOM  FROB) 

Value  Is  NIL  because  It  Is  within  false  branch  of  the  cond  clause 

((ATOM  FROB) 

(COND  ((NULL  FROB)  NIL) 

(T  ((LAMBDA  (X-3)  (CONS  X-3  X-3 ) )  (CAT  FROB))))) 
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((OR  T  (CDR  FROB) )  (CAR  FROB)) 

Version:  10  Expression  type:  *ATOM*  Transformer:  SEXPR ! FORM-BVAL 


:u 


(CONO  ((ATOM  FROB) 

(COND  ((NULL  FROB)  NIL) 

(T  ((LAMBDA  (X-3)  (CONS  X-3  X-3 ) )  (CAT  FROB))))) 

((OR  T  (CDR  FROB))  (CAR  FROB)) 

(T  (CAR  (FOO  FROB)))) 

Version:  10  Expression  type:  *AT0M«  Transformer:  SEXPR ! FORM-BVAL 

:  + 


(COND  ((ATOM  FROB) 

(CONO  ((NULL  FROB)  NIL) 

(T  ((LAMBDA  (X-3)  (CONS  X-3  X-3))  (CAT  FROB))))) 
((OR  T)  (CAR  FROB)) 

(T  (CAR  (FOO  FROB)))) 

Version:  11  Expression  type:  *OR»  Transformer:  ORITRIM 

:q 


After  walking,  the  user  is  returned  to  the  transformer  activities  menu.  From  there  he  may  either 
inquire  more  about  the  same  transformer,  or  return  to  the  transformer  menu. 


TRANSFORMER  ACTIVITIES  MENU 

0  -  Return  to  Transformer  Menu 

1  -  Observe  all  1  attempts  of  transformer  ORITRIM 

2  -  Only  its  1  successful  applications 

3  -  Only  its  0  unsuccessful  applications 
Enter  Q  to  quit. 

Selection?  (end  with  CR):  0 

TRANSFORMER  MENU 
0  -  Return  to  Menu  Menu 

1  -  Transformers  given  In  transformation  sets 

2  -  Transformers  attempted 

3  -  Transformers  which  failed 

4  -  Transformers  which  succeeded 
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Enter  Q  to  quit. 

Enter  transformer  iame  to  view. 
Selection?  (end  with  Cl):  condlsimplify 


TRANSFORMER  ACTIVITIES  MENU 
0  -  Return  to  Transformer  Menu 

1  -  Observe  all  3  attempts  of  transformer  CONDISIMPLIFY 

2  -  Only  its  0  successful  applications 

3  -  Only  its  3  unsuccessful  applications 
Enter  Q  to  quit. 

Selection?  (end  with  CR):  3 

The  transformer  CONDISIMPLIFY  failed  1  times  during  transformation  time  3. 
Would  you  like  to  see  the  expressions?  (enter  y  or  n)  j 

If  the  user  chooses  to  view  an  expression  to  which  a  transformer  was  applied  but  failed,  after 
displaying  the  expression,  the  names  of  all  transformers  which  were  attempted  (but  foiled)  at  that 
time  arc  displayed  along  with  their  justifications. 


Expression  1  of  1.  Continue?  (enter  y  or  n)  y 

(COND  ((NULL  Y)  NIL)  (T  ((LAMBDA  (X-l)  (CONS  X-l  X-l))  (CAT  Y)))) 

Version:  3  Expression  type:  *COND*  Transformer:  C0ND1CREATE 

Transformation  time:  3  Expression  type:  »C0ND» 

CONDI  TRIM  failed  to  transform  the  expression  because 
no  predicates  were  known  (maybe  last). 

CONDISIMPLIFY  failed  to  transform  the  expression  because 
patterns  didn't  match. 

SEXPRl FORM-BVAL  failed  to  transform  the  expression  because 
the  boolean  value  is  unknown. 

Oone. 

Enter  any  character  to  continue:  x 


The  transformer  CONDISIMPLIFY  failed  1  times  during  transformation  time  6. 
Would  you  like  to  see  the  expressions?  (enter  y  or  n)  n 

The  transformer  CONDISIMPLIFY  failed  1  times  during  transformation  time  12. 
Would  you  like  to  see  the  expressions?  (enter  y  or  n)  y 
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Expression  1  of  1.  Continue?  (enter  y  or  n)  y 


(COND  ((ATOM  FROB) 

(COND  ((NULL  FROB)  NIL) 

(T  ((LAMBDA  (X-3)  (CONS  X-3  X-3))  (CAT  FROB))))) 

(T  (CAR  FROB))) 

Version:  14  Expression  type:  »C0ND«  Transformer:  C0ND1TRIM 
Transformat  ion  time:  12  Expression  type:  »COND* 

CONOISIMPLIFY  failed  to  transform  the  expression  because 
patterns  didn't  match. 

SEXPR ! FORM-BVAL  failed  to  transform  the  expression  because 
the  boolean  value  is  unknown. 

Done. 

Enter  any  character  to  continue:  x 

If  the  user  quits  from  any  menu,  he  leaves  the  menu  system  and  is  returned  to  die  driver  function, 
which  allows  him  to  write  out  the  transformed  expressions. 


TRANSFORMER  ACTIVITIES  MENU 
0  -  Return  to  Transformer  Menu 

1  -  Observe  all  3  attempts  of  transformer  COND 1  SIMPLIFY 

2  -  Only  its  0  successful  applications 

3  -  Only  its  3  unsuccessful  applications 
En  „ ir  Q  to  quit. 

Selection?  (end  with  CR):  q 

Write  final  output  to  a  file?  (enter  y  or  n)  y 
Enter  file  name:  bkerns:xrormd > 

Written. 

T 

The  file  bkerns ;  xformd  >  now  contains: 

(DEFUN  TEST  (X  Y) 

(COND  ((NULL  Y)  NIL) 

(T  ((LAMBDA  (X-l)  (CONS  X-l  X-l))  (CAT  Y))))) 
(DEFUN  F00  (X)  (CONS  X  X)) 


(DEFUN  BAR  (FROB) 


COND  ((ATOM  FROB) 

(COND  ((NULL  FROB)  NIL) 
(T  ((LAMBDA  (X-3) 
(T  (CAR  FROB)))) 
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n  r  works.  The  code  has  run  successfully  with  the  applicable  LISP  programs  given  it,  and  the 
set  of  optimizing  transformers  I  have  written  to  demonstrate  the  system  have  even  managed 
to  significantly  improve  the  execution  time  of  those  programs,  That  is  not  to  say  that  the  sy  stem  was 
all  1  envisioned,  or  that  it  cannot  be  more . . . 


5.1.  Looking  Back 

It  was  my  intention  from  the  beginning  that  the  transformational  component  be  aide  to  run 
independently  of  the  accountable  component  (which  in  my  mind  meant  independently  of  the  cell 
representation),  simply  by  rewriting  the  definitions  of  XFORM  and  RESPOND.  Clearly,  the  func¬ 
tions  which  access  pieces  of  the  internal  form  must  be  aware  of  the  representation  of  that  form,  but 
these  functions  can  be  written  to  accept  any  of  a  finite  number  of  different  representations  at  any 
time,  so  long  as  they  are  able  to  figure  out  what  that  representation  is  by  means  of  checking  type 
flags. 

I  still  believe  this  is  possible;  however,  it  is  clear  that  the  system  as  currently  implemented 
is  not  able  to  run  with  an  internal  form  as  simple  as  just  the  LISP  parse  tree.  Certain  contextual 
information  is  necessary  for  correct  transformation.  Without  the  predicate  position  information 
currently  stored  in  the  internal  form  of  the  parse  tree,  transformers  dependent  on  that  informa¬ 
tion  cannot  apply.  Without  back-pointer  information,  for  example,  it  is  impossible  for  a  slave 
transformer  to  delete  its  input  expression;  it  has  no  access  to  the  father  expression  from  which 


94 


Chapter  Five 


Conclusions 


to  unhook  it.  Although  all  such  information  might  have  been  created  and  maintained  in  vari¬ 
ables  separate  form  the  internal  form  and  passed  from  control  function  to  control  function  (as 
the  environment  and  index  of  bindings  were),  some  of  it  was  more  easily  stored  directly  along 
with  the  LISP  code  to  which  it  pertained.  Thus,  some  sort  of  augmented  parse  tree  is  necessary. 
Furthermore,  the  current  system  is  dependent  on  the  fact  that  XFORM  will  install  the  returned 
expression  in  the  internal  form  (whatever  it  is)  by  side-effecting  that  form,  since  many  system 
functions  do  not  “hold  on"  to  their  values.  This  means  that  the  transformation  system  in  its  present 
implementation  cannot  transform  itself,  a  fact  I  truly  regret. 

It  was  my  further  intention  that  the  accountable  system  be  implcmcntable  regardless  of  the 
nature  of  the  implementation  of  the  transformational  component.  While  this  is  true  to  some  degree, 
there  are  nevertheless  dependencies  1  saw  no  simple  way  of  avoiding.  Some  of  them  have  been 
stated  already  in  Chapter  Three:  the  independence  of  transformations  from  the  control  structure, 
the  restriction  that  transformations  are  called  via  XFORM  and  return  via  RESPOND,  etc.  In  addi¬ 
tion,  in  order  for  cell  histories  to  be  properly  maintained,  slave  transformers  must  be  used  to  keep 
transformations  as  localized  as  possible.  Consider  the  result  of  consing  up  a  new  list  of  clauses  to  be 
returned  by  the  transformation: 

(CONO  <clauses-l>  (T  (COND  <clauses-2>))) 

=>  (COND  <clauses-l>  <clauses-2>) 

First  of  all,  since  optimizing  transformations  are  required  to  return  equivalent  expressions,  the 
transformer  would  have  to  return  a  conditional  expression.  Thus,  after  constructing  the  new  list  of 
clauses  (perhaps  by  appending  <clauses-l>  and  <c1  auses-2>),  it  creates  a  new  conditional 
expression  via  COND !  CREATE,  which  takes  a  list  of  clauses  and  returns  a  conditional  expression. 
This  function  will  result  in  the  creation  of  a  new  *COND*  cell;  when  XFORM  realizes  that  the 
returned  expression  is  neither  the  same  nor  a  subcell  of  the  input  cell,  it  will  correctly  assume  that 
the  returned  cell  is  new.  The  new  cell's  subtree  (the  list  of  clauses)  will  be  installed  as  the  input 
cell’s  new  monk,  but  only  after  the  subcclls  of  the  input  conditional  have  been  terminated.  Since 
these  same  subcclls  arc  contained  in  the  newly  conscd  list  of  clauses,  the  resulting  *COND*  cell 
consists  of  an  empty  clause  list 

This  example  illustrates  some  of  the  dependencies  of  the  accountable  component  on  the  trans¬ 
formation  implementor.  The  implementor  needs  to  be  aw  are  of  the  five  different  cases  into  which 
returned  objects  may  fall,  and  write  his  transformers  accordingly.  Correctly  maintaining  the  cell 
histories  is  tricky  business,  and  I  have  not  yet  succeeded  in  making  the  mechanism  robust  enough 
to  survive  even  the  good  intentions  of  uninformed  transformation  implementors. 
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I  spent  the  bulk  of  my  effort  for  this  thesis  in  gathering  information  and  developing 
mechanisms  for  recording  it.  rather  than  using  that  information.  1  believe  that  future  work  on  this 
system  should  emphasize  the  analysis  of  the  transformation  process  using  the  information  gathered 
by  the  recording  element.  1  dream  of  an  accountable  system,  for  example,  that  could  reply  to 
the  question  “Why  didn’t  this  simplifying  transformer  apply?”  with  an  answer  something  like 
“Hmmm,  if  only  1  could  have  shown  the  predicate  <prcdicate>  to  be  false,  I  could  have  completely 
eliminated  the  computation  <hairy  exprcssion>”.  Or  to  the  question  “Why  did  this  transformer 
apply?”,  reply  “if  it  hadn’t  been  for  this  assertion  you  coded  in  by  hand,  none  of  this  whole  reduc¬ 
tion  would  have  happened.”  That  is,  the  query  element  could  not  only  report  the  facts,  but  perform 
some  sort  of  analysis  using  those  facts.  It  involves  more  carefully  itemizing  the  prerequisites  of  a 
transformation  and  understanding  exactly  which  of  them  were  met  (and  why),  and  which  were  not. 
The  “why”  needs  to  be  slightly  more  sophisticated  than  the  justifications  of  the  present  system;  if 
a  prerequisite  is  met,  the  system  should  understand  the  source  of  the  information  which  satisfied 
that  prerequisite.  Was  it  an  assertion,  deduced  from  context,  or  always  true,  and  then,  what  is  the 
significance  of  that  source? 

If  a  transformation  did  not  apply,  then  how  close  did  it  come?  To  answer  this  question  the 
system  must  know  not  only  which  prerequisites  failed,  but  be  able  to  suggest  ways  of  satisfying 
them.  Would  an  assertion  solve  the  problem,  or  would  such  an  assertion  be  a  contradiction  to  cur¬ 
rent  information.  If  a  transformation  failed  because  it  tried  to  simplify  an  OR  expression  with  one 
argument,  for  example,  but  was  applied  to  an  OR  expression  with  two  arguments,  simply  asserting 
that  the  expression  had  only  one  arguments  would  raise  a  contradiction. 

And  finally,  though  I  would  enjoy  providing  a  super  slick  user  interface  for  the  accountable 
transformation  system,  such  improvements  are  not  as  dependent  on  the  implementation  of  this 
system  so  much  as  they  arc  on  the  capabilities  of  the  user's  terminal  and  its  host  system.  Just  the 
same,  1  dream  of  display  hacks  which  use  multiple  cursors  and  can  remember  where  a  particular 
s-expression  is  on  the  screen.  Then  instead  of  having  to  redisplay  an  expression  to  refer  to  it, 
the  cursor  simply  jumps  to  the  correct  s-expression  already  displayed.  When  the  user  asks  to  see 
the  next  version  of  the  displayed  expression,  the  cursor  jumps  to  the  sub-expression  about  to  be 
transformed,  waits  for  a  signal,  then  pops  in  the  new  sub-expression.  A  status  line  on  die  bottom 
of  the  screen  keeps  the  user  informed  as  to  the  current  version,  expression  type,  and  transformer 
name.  Then  of  course,  menus  pop  up  on  the  screen  and  a  mouse  is  available  to  control  the  cursor. 
However,  1  will  leave  the  implementation  of  such  features  to  hackers  equipped  with  die  necessary 
terminals. 
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5.3.  The  Present 

As  is  always  the  case,  I  have  not  accomplished  all  I  had  set  out  to  do  in  this  thesis.  Though 
I  believe  the  present  implementation  qualifies  as  an  accountable  system,  it  is  not  what  1  proposed 
that  it  be  originally:  a  responsible  system.  To  be  responsible  for  its  actions,  it  must  have  a  clearer 
understanding  of  those  actions  and  be  able  to  analyze  a  situation  to  the  extent  that  it  can  propose 
to  the  user  the  course  of  action  necessary  to  correct  any  problems  in  the  transformation  process.  An 
accountable  system,  however,  is  a  step  forward,  and  does  provide  a  hitherto  undeveloped  service. 
It  allows  the  user  to  observe  the  sequence  of  transformations  applied  and  study  their  interactions 
with  each  other,  it  allows  him  to  select  any  portion  of  code  in  any  point  in  time,  and  it  allows  him  to 
control  the  flow  of  information  returned  to  him  at  his  request 
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